Three-dimensional reconstruction of anatomy from sparse medical imaging data

Abstract

Anatomical modeling systems, methods, and devices are disclosed herein. According to some variations, a method of the present technology includes receiving an incomplete three- dimensional (3D) model of an anatomy of interest of a subject. The method can further include inputting the incomplete 3D model into a pre-trained machine learning algorithm, wherein the machine learning algorithm encodes a latent space representation of the anatomy of interest from training data derived from a plurality of seconds. The method can also include generating, using the machine learning algorithm, an expanded 3D model of the anatomy of interest of the subject. Methods for training the machine learning algorithm are also disclosed.

Description

THREE-DIMENSIONAL RECONSTRUCTION OF ANATOMY FROM SPARSEMEDICAL IMAGING DATACROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority to U.S. Provisional App. No. 63 / 638,774, filed April 25, 2024, which is incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] The present technology relates to three-dimensional (3D) reconstruction of anatomy from sparse medical imaging data.BACKGROUND

[0003] Medical imaging modalities and systems enable medical practitioners to design and implement treatment plans customized to an individual subject’s needs, anatomy, and current conditions. In a typical medical imaging procedure, a subject’s anatomy is reconstructed into two-dimensional (2D) and / or three-dimensional (3D) models based on detected signals (e.g., a distribution of a radiotracer within a subject). In some instances, the reconstructed models can be displayed in real-time and provide image-based guidance intraoperatively, such as for cardiac implant procedures. The reconstructed models can also be reviewed pre-operatively or post-operatively to determine a suitable course of action for a given subject.

[0004] Unfortunately, issues in medical imaging procedures can result in reconstructed models that are insufficient, missing data, and / or incomplete. For example, movement of the subject and / or imaging apparatus can produce signal artefacts (e.g., blurring). As another example, imaging windows (e.g., field of view) may be limited based at least in part on placement of an imaging probe. As another example, imaging capabilities may be limited at least in part by structures (e.g., bones or other tissue not of interest) around an anatomy of interest. These challenges and more can diminish the accuracy of the reconstructed models and limit a medical practitioner’s available information.SUMMARY

[0005] The subject technology relates to systems and methods for generating a 3D reconstruction of anatomy from sparse imaging of the anatomy. For example, the subjecttechnology can help improve the accuracy and / or completeness of a 3D reconstruction of anatomy when an initial 3D reconstruction is limited at least in part due to partial imaging data of the anatomy.

[0006] The subject technology is illustrated, for example, according to various aspects described below, including with reference to FIGS. 1-9. Various examples of aspects of the subject technology are described as numbered clauses (1, 2, 3, etc.) for convenience. These are provided as examples and do not limit the subject technology.1. A method compri sing : receiving an incomplete three-dimensional (3D) model of an anatomy of interest of a subject; inputting the incomplete 3D model into a pre-trained machine learning algorithm, wherein the machine learning algorithm encodes a latent space representation of the anatomy of interest from training data derived from a plurality of training subjects; and generating, using the machine learning algorithm, an expanded 3D model of the anatomy of interest of the subject.2. The method of clause 1, wherein the expanded 3D model is directly outputted from the machine learning algorithm.3. The method of clause 1 or 2, wherein the incomplete 3D model comprises a first model portion corresponding to a first anatomical region and generating the expanded 3D model includes outputting, from the machine learning algorithm, a second model portion corresponding to a second anatomical region.4. The method of any one of clauses 1-3, wherein the training data comprises results from applying statistical shape modeling on a plurality of 3D models of the anatomy of interest corresponding to the plurality of training subjects.5. The method of any one of clauses 1^1, wherein the machine learning algorithm comprises an artificial neural network.6. The method of clause 5, wherein the artificial neural network comprises one or more of a convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), capsule network (CapsNets), graph neural network (GNN), encoder-decoder network, vision transformer, or Siamese neural network.7. The method of any one of clauses 1-6, wherein the latent space representation includes a lower-dimensional representation of the anatomy of interest.8. The method of any one of clauses 1-7, wherein the latent space representation is learned using principal component analysis.9. The method of any one of clauses 1-8, wherein the latent space representation includes one or more latent variables related to at least one of a size or geometry of the anatomy of interest.10. The method of any one of clauses 1-9, wherein the anatomy of interest is a heart.11. The method of any one of clauses 1-10, wherein the incomplete 3D model is reconstructed from one or more of fluoroscopy, ultrasound, X-ray, computed tomography, positron emission tomography, magnetic resonance imaging, or single-photon emission computed tomography.12. The method of any one of clauses 1-11, further comprising displaying the expanded 3D model on a display.13. The method of clause 12, wherein displaying the expanded 3D model comprises displaying a first portion of the anatomy of interest in accordance with a first display scheme, and displaying a second portion of the anatomy of interest in accordance with a second display scheme.14. A system comprising: a processor; and a memory operably coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving, via the processor, an incomplete 3D model of an anatomy of interest of a subject; inputting, via the processor, the incomplete 3D model into a pre-trained machine learning algorithm, wherein the machine learning algorithm encodes a latent space representation of the anatomy of interest from training data derived from a plurality of training subjects; and generating, via the processor, using the machine learning algorithm, an expanded 3D model of the anatomy of interest of the subject.15. The system of clause 14, wherein the expanded 3D model is directly outputted from the machine learning algorithm.16. The system of clause 14 or 15, wherein the incomplete 3D model comprises a first model portion corresponding to a first anatomical region and generating the expanded 3D model includes outputting, via the processor, from the machine learning algorithm, a second portion corresponding to a second anatomical region.17. The system of any one of clauses 14-16, wherein the training data comprises results from applying statistical shape modeling on a plurality of 3D models of the anatomy of interest corresponding to the plurality of training subjects.18. The system of any one of clauses 14-17, wherein the machine learning algorithm comprises an artificial neural network.19. The system of clause 18, wherein the artificial neural network comprises one or more of a convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), capsule network (CapsNets), graph neural network (GNN), encoder-decoder network, vision transformer, or Siamese neural network.20. The system of any one of clauses 14-19, wherein the latent space representation includes a lower-dimensional representation of the anatomy of interest.21. The system of any one of clauses 14-20, wherein the latent space representation is learned using principal component analysis.22. The system of any one of clauses 14-21, wherein the latent space representation includes one or more latent variables related to at least one of a size or geometry of the anatomy of interest.23. The system of any one of clauses 14-22, wherein the anatomy of interest is a heart.24. The system of any one of clauses 14-23, wherein the incomplete 3D model is reconstructed from one or more of fluoroscopy, ultrasound, X-ray, computed tomography, positron emission tomography, magnetic resonance imaging, or single-photon emission computed tomography.25. The system of any one of clauses 14-24, further comprising a display configured to display the expanded 3D model.26. The system of clause 25, wherein the instructions that, when executed by the processor, cause the system to display the expanded 3D model comprises instructions that, when executed by the processor, cause the system to perform operations comprising displaying a first portion of the anatomy of interest in accordance with a first display scheme, and displaying a second portion of the anatomy of interest in accordance with a second display scheme.27. The system of any one of clauses 14-26, further comprising an imaging system.28. The system of clause 27, wherein the imaging system comprises an imaging probe.29. The system of clause 28, wherein the imaging probe comprises an electromagnetic tracking sensor.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale. Instead, emphasis is placed on illustrating clearly the principles of the present disclosure.

[0008] FIG. l is a schematic diagram of an example computing environment including an anatomical modeling system, in accordance with variations of the present technology.

[0009] FIG. 2 is a flow chart of an example method for training a machine learning algorithm of an anatomical modeling system, in accordance with variations of the present technology.

[0010] FIG. 3 is a flow chart of an example method for training a machine learning algorithm of an anatomical modeling system, in accordance with variations of the present technology.

[0011] FIG. 4 is a schematic diagram of an example machine learning architecture of an anatomical modeling system, in accordance with variations of the present technology.

[0012] FIGS. 5 A and 5B are schematic diagrams of a node correspondence process for an example statistical shape model, in accordance with variations of the present technology.

[0013] FIG. 6 is a flow chart of an example method for anatomical modeling, in accordance with variations of the present technology.

[0014] FIG. 7 is a schematic diagram of an example machine learning architecture of an anatomical modeling system, in accordance with variations of the present technology.

[0015] FIG. 8 is a schematic illustration of an example display of an anatomical model formed in accordance with variations of the present technology.

[0016] FIG. 9 is a schematic diagram of an example machine learning architecture of an anatomical modeling system, in accordance with variations of the present technology.DETAILED DESCRIPTION

[0017] The present technology relates to systems, methods, and devices for generating a complete reconstruction of an anatomy of interest of a subject from sparse and / or incompletemedical imaging data. Some variations of the present technology, for example, are directed to an anatomical modeling system that uses a machine learning algorithm trained on a plurality of anatomical models to generate anatomical portions. The anatomical modeling system can, for example, receive an incomplete 3D model of an anatomy of interest of a subject and generate an expanded 3D model of the anatomy of interest. In some variations, the incomplete 3D model can include missing and / or inaccurate data which are replaced in the expanded 3D model. Specific details of several variations of the present technology are described below with reference to FIGS. 1-9.[0018| The anatomical modeling systems, devices, and methods of the present technology can be used in pre-operative, intraoperative, and / or post-operative procedures and for a variety of subject conditions and anatomies of interest. In some variations, the anatomical modeling system can be used to predict portions of an anatomy of interest that were insufficiently captured and / or omitted from a medical imaging procedure. For example, the anatomical modeling system can analyze a partially-reconstructed model where the partially- reconstructed model is missing one or more portions of the anatomy of interest, and generate a fully-reconstructed model including such one or more anatomical portions of interest. While the anatomical modeling systems and methods are largely described herein with respect to generating a reconstruction of a heart (that is, with the heart as the anatomy of interest), it should be understood that the anatomical modeling systems and methods described herein can additionally or alternatively be applied to reconstruct any tissue of interest, including other organs (e.g., brain, lung, kidney, liver, bone), and / or other suitable tissues.[00191 The present technology can provide many advantages for image reconstruction. For example, predicting portions of an anatomy of interest that were insufficiently captured can save time and resources associated with additional procedures. Furthermore, the predicted anatomy of interest can reduce the amount of guesswork required by the medical provider to navigate the anatomy of interest. Additionally, the anatomical modeling system described herein can be used retroactively, such as when a subject is no longer in the hospital and is unavailable for further examination. The anatomical modeling system can also assist medical providers in intraoperative procedures where full medical imaging may be impossible due to physical obstructions and / or safety limitations. Further advantages will be illustrated with respect to various variations of the anatomical modeling system, as detailed herein.I. ANATOMICAL MODELING SYSTEMS AND METHODS

[0020] The model reconstruction systems described herein can be used as part of a computing environment. For example, FIG. 1 is a schematic diagram of a computing environment 100 in which an anatomical modeling system 102 operates in accordance with variations of the present technology. As shown in FIG. 1, the anatomical modeling system 102 is operably coupled to one or more imaging systems 104 via a network 106. Additionally, the anatomical modeling system 102 is operably coupled to at least one database or storage component 108 (“database 108”). The anatomical modeling system 102 can be or include a server or other computing system or device having processors, memory, software and / or hardware components configured to implement the various methods described herein. In some variations, the anatomical modeling system 102 is implemented as a distributed “cloud” computing system across any suitable combination of hardware and / or virtual computing resources. Although the imaging system 104 and image reconstruction system 102 are illustrated schematically in FIG. 1 as separate devices or components, in some variations these components can be embodied in a single device.

[0021] As described in greater detail herein, the anatomical modeling system 102 can be configured to receive, process, and / or generate 3D models of subject anatomy. The 3D models can include coordinate-based representations of anatomical structures of a subject. The 3D models can be constructed in simulated environments via computer modeling systems, biomechanical systems or apparatus, and the like (e.g., 3D medical visualization software). The 3D models can also be finite element models created using computer program application software. For instance, solid geometry 3D models can be created using computer aided engineering (CAE) or computer aided design (CAD) programs such as the AutoCAD® software products available from Autodesk, Inc., of San Rafael, Calif. The 3D models can include both anatomical models (e.g., a heart) and medical device models (e.g., a pacemaker). The 3D models can be received, processed, and / or generated by the imaging system 102, the anatomical modeling system 104, the network 106, or a suitable combination thereof.

[0022] The anatomical modeling system 102 can include one or more processors 102a, and one or more memory devices 102b having instructions stored therein. The one or more memory device 102b can include any suitable computer-readable medium such as RAMs, ROMs, flash memory, EEPROMs, optical devices (e.g., CD or DVD), hard drives, floppy drives, or any suitable device. The memory device 102b can include instructions (e.g.,organized in one or more modules) for performing 3D model generation in accordance with any of the methods described in further detail herein.

[0023] The processor 102a may be configured to execute the instructions that are stored in the memory device 102b such that, when it executes the instructions, the processor 102a perform aspects of the methods herein. The instructions may be executed by computerexecutable components integrated with a software application, applet, host, server, network, website, communication service, communication interface, hardware, firmware, software elements of a user computer or mobile device, smartphone, or any suitable combination thereof. In some variations, the one or more processors 102a can be incorporated into a computing device or system such as a cloud-based computer system, a mainframe computer system, a grid-computer system, or other suitable computer system.

[0024] In some variations, the anatomical modeling system 102 receives 3D models from and / or transmit 3D models to the imaging system 104. Optionally, the anatomical modeling system 102 can receive medical imaging data and convert the medical imaging data into 3D models. Although FIG. 1 illustrates a single imaging system 104, in other variations the anatomical modeling system 102 can be connected to a plurality of imaging systems 104. The imaging system 104 can include imaging apparatuses and like components needed for collecting imaging data using ultrasound, fluoroscopy, X-ray, computed tomography (CT), positron emission tomography (PET), magnetic resonance imaging (MRI), single-photon emission computed tomography (SPECT), diagnostic sonography, or a combination thereof. From the imaging data, the imaging system 104 and / or the anatomical modeling system 102 can generate 2D or 3D models of anatomy. The 2D or 3D models of anatomy can be generated automatically or semi-automatically by the imaging system 104, or manually via segmentation by a user. For instance, the imaging system 104 may convert the imaging data into 2D slices and / or 3D models. As will be understood by a person having ordinary skill in the art, many different types of medical imaging modalities and imaging systems 104 may be used in combination with the anatomical modeling system 102.

[0025] In some variations, the anatomical modeling system 102 is configured to evaluate an incomplete 3D model of an anatomy of interest of a subject and generate an expanded 3D model of the anatomy of interest of a subject using a machine learning algorithm. The machine learning algorithm can include supervised learning models, unsupervised learning models, semi-supervised learning models, and / or reinforcement learning models. Examples of machine learning models suitable for use with the present technology include, but are notlimited to: regression algorithms, instance-based algorithms, regularization algorithms, decision tree algorithms, Bayesian algorithms, clustering algorithms, association rule learning algorithms, artificial neural networks, deep learning algorithms (e.g., convolutional neural networks, recurrent neural networks, long short-term memory networks, stacked autoencoders, Siamese neural networks, generative adversarial networks, capsule networks, graph neural networks, encoder-decoder networks, deep Boltzmann machines, deep belief networks, transformers, vision transformers, language models), principal component analysis (PCA), singular value decomposition (SVD), other dimensionality reduction algorithms, time series forecasting algorithms, and ensemble algorithms.

[0026] The anatomical modeling system 102 can be operably coupled to the database 108. In some variations, the database 108 includes training data for the machine learning algorithm. As will be described further herein, the training data can include a collection (e.g., repository) of 3D models of anatomy of interest from a plurality of subjects, one or more metrics of anatomy of interest from a plurality of subjects (e.g., dimensional measurements of the anatomy of interest), and / or other characteristics associated with a plurality of subjects (e.g., demographics). The training data can include identification data such as “Subject 1” and “Heart Model.” Alternatively, or in combination, the database 108 can include statistical shape models (SSMs), as will be further described herein. Various examples of training data are described in further detail herein. In some variations, the database 108 is updated by the imaging system 104, the anatomical modeling system 102, and / or the network 106. For example, new imaging data from the imaging system 104 can result in new 3D models being stored in the database 108.

[0027] The anatomical modeling system 102 and imaging system 104 can be operably and communicatively coupled to each other via the network 106. The network 106 can be or include one or more communications networks and can include at least one of the following: a wired network, a wireless network, a metropolitan area network (“MAN”), a local area network (“LAN”), a wide area network (“WAN”), a virtual local area network (“VLAN”), an internet, an extranet, an intranet, and / or any other type of network and / or any combination thereof. Additionally, although FIG. 1 illustrates the anatomical modeling system 102 as being directly connected to the database 108 without the network 106, in other variations the anatomical modeling system 102 can be indirectly connected to the database 108 via the network 106. Additionally, or alternatively, one or more of the imaging systems 104 can be configured tocommunicate directly with the anatomical modeling system 102 and / or database 108, rather than communicating with these components via the network 106.[00281 The various components illustrated in FIG. 1 (e.g., the anatomical modeling system 102, imaging system 104, network 106, database 108) can include any suitable combination of hardware and / or software. In some variations, these components can be disposed on one or more computing devices, such as, server(s), database(s), personal computer(s), laptop(s), cellular telephone(s), smartphone(s), tablet computer(s), and / or any other computing devices and / or any combination thereof. In some variations, these components can be disposed on a single computing device and / or can be part of a single communications network. In some variations, the components can be located on distinct and separate computing devices.A. TRAINING AN ANATOMICAL MODELING SYSTEM[00291 FIG. 2 is a flow diagram illustrating a method 200 for training a machine learning algorithm of an anatomical modeling system (e.g., anatomical modeling system 102), in accordance with variations of the present technology. The machine learning algorithm, once trained, can be configured to receive an incomplete 3D model of an anatomy of interest of a subject and generate an expanded 3D model of the anatomy of interest of the subject. In some variations, the incomplete 3D model includes a first portion corresponding to a first anatomical portion, and the expanded 3D model can include one or more additional portions corresponding to additional anatomical portions. The method 200 can be performed using any of the systems and devices described herein, such as the anatomical modeling system 102 of FIG. 1. In some variations, some or all of the processes of the method 200 are implemented as the result of computer-readable instructions (e.g., program code) that are configured to be executed by one or more processors (e.g., processor 102a).

[0030] The training method 200 can include receiving a first plurality of three- dimensional (3D) models of an anatomy of interest 202 as training data. In some variations, the first plurality of 3D models are reconstructed from medical imaging (e.g., CT, MRI, ultrasound, fluoroscopy). The reconstruction can be performed by an anatomical modeling system (e.g., the anatomical modeling system 102 of FIG. 1), an imaging system (e.g., the imaging system 104 of FIG. 1), and in a local and / or remote manner (e.g., over the network 106 of FIG. 1). Each of the first plurality of 3D models can correspond to medical imaging data associated with one of a plurality of training subjects. For example, the first plurality of 3Dmodels can include a 3D model reconstructed from first medical imaging data from a first subject and a second 3D model reconstructed from second medical imaging data from a second subject. The first medical imaging data and the second medical imaging data can correspond to a particular anatomical structure (e.g., respective hearts of the first subject and the second subject). In some variations, the first plurality of 3D models can include generally complete or full 3D models of the anatomy of interest (e.g., a full heart), incomplete or partial 3D models of the anatomy of interest, or a combination thereof. Accordingly, the machine learning algorithm of an anatomical modeling system (e.g., anatomical modeling system 102) can be trained using full 3D models, partial 3D models, or a combination thereof.

[0031] The first plurality of 3D models can be reconstructed from medical imaging data stored in a database, where the medical imaging data corresponds to any suitable number of subjects. For example, the first plurality of 3D models can include 3D models reconstructed from at least 2 subjects, at least 5 subjects, at least 20 subjects, at least 100 subjects, at least 500 subjects, at least 1000 subjects, at least 10,000 subjects, etc. In some variations, receiving data from a greater number of subjects can increase the accuracy of the machine learning algorithm. Still, in some variations, receiving data from fewer subjects may reduce computational time and improve efficiency of the anatomical modeling system 102.

[0032] In some variations, training method 200 may include receiving other information characteristic of anatomy of interest among the training subjects and / or information relating to characteristics of training subjects providing the 3D models. For example, in variations in which the anatomy of interest is or includes a heart, the training data may include dimensional measurements of anatomy around the heart (e.g., chest circumference) and / or the heart itself (e.g., right ventricle axis length, left ventricle axis length, volume and / or width of one or more heart chambers (left atrium, right atrium, left ventricle, right ventricle, etc.), diameter or other dimension(s) of one or more heart valves (aortic valve, mitral valve, etc.), and / or other suitable anatomical measurements alone or in combination providing numerical characterization of the anatomy of interest.

[0033] The subjects from which the first plurality of 3D models are derived can share one or more characteristics. For example, the subjects may share similar cardiac profiles as determined by medical examination (e.g., generally similar heart sizes, heart chamber sizes, heart chamber shapes, any heart conditions, etc.). The subjects may additionally or alternatively have similar demographics (e.g., age, sex, height, weight, BMI, etc.), lifestyles, and / or medical conditions (e.g., disease state). Alternatively, the subjects may include subjects havingdissimilar cardiac profiles, demographics, lifestyles, and / or medical conditions. The selection of the first plurality of 3D models, and by proxy, the selection of the subjects for training data, may depend on the desired use-case of the machine learning algorithm (e.g., intended patients for which anatomy modeling will be performed). For example, for a particular instance of the machine learning algorithm, the subjects from which the training data is derived can correspond in sex, age, weight, height, BMI, and / or medical condition, etc. to the patient population intended to be modeled using the trained machine learning algorithm. For example, a particular instance of the machine learning algorithm intended for reconstructing an anatomy of interest for females may be trained using a first plurality of 3D models derived from entirely female subjects. As such, training data for training a machine learning algorithm of an anatomical modeling system may include information relating to suitable characteristics of the training subjects.

[0034] The method 200 can further include analyzing the first plurality of 3D models 204 to encode a latent space of the first plurality of 3D models, where the latent space comprises a plurality of latent variables. In some variations, analyzing the first plurality of 3D models includes a processing stage and a learning stage. Processing the first plurality of 3D models can include, but is not limited to, noise reduction, contrast enhancement, segmentation, registration, motion analysis, structural analysis, cropping, resampling, and / or filtering, etc. For example, processing of the first plurality of 3D models may include running an artefact reduction algorithm on the first plurality of 3D models. The learning stage can include data compression, dimensionality reduction, downsampling, decimation, resampling, feature selection, feature extraction, wrapping, embedding, and / or further processing as described herein.

[0035] In some variations, analyzing the first plurality of 3D models 204 includes applying a Statistical Shape Model (SSM) to the first plurality of 3D models. SSMs provide a way of characterizing a given shape based on the shape’s population variation and mean. For example, an SSM applied to a collection of like objects (e.g., type of anatomy, such as a heart) provides a statistical evaluation of the general shape of the object and how the objects differ from one another. SSMs can be used for classification, segmentation, and phantom generation (e.g., producing new representations of the object). For instance, the SSMs can increase the number of 3D models included in the first plurality of 3D models by interpolating and / or extrapolating data from the existing 3D models of the first plurality of 3D models. In some variations, applying an SSM to the first plurality of 3D models can increase the number of 3Dmodels in the first plurality of 3D models by a factor of 1.1, 1.5, 2, 5, 10, 20, 100, etc. The SSMs described herein can be used to generate training data for machine learning algorithms.

[0036] Applying the SSM can include transforming the first plurality of 3D models to improve ease and / or accuracy of comparison. The transformation can include characterizing each 3D model of the first plurality of 3D models using node correspondence. Node correspondence includes labeling features consistently across 3D models. For example, when the 3D models include reconstructions of hearts, node correspondence may include segmenting (e.g., partitioning the 3D models into nodes, or discrete groups of model segments) and / or labeling anatomical features such as at least one of the arch of the aorta, Bachman’s bundle, left atrium, atrioventricular (AV) bundle (bundle of His), left ventricle, left bundle branch, Purkinje fibers, right bundle branch, right ventricle, right atrium, posterior internodal, middle internodal, AV node, anterior internodal, sinoatrial (SA) node, ventricular apex, veins, valves, trunks, and / or other suitable cardiac anatomy. Alternatively, or in combination, the nodes can be mathematical nodes determined by a mathematical or geometrical property. For example, the nodes can reflect a degree of curvature, points of inflection, local or global maxima or minima, crevasses, ridges, protrusions, indentations, sub-geometries, etc. The number of nodes used for node correspondence can vary depending on the details and surface morphology of the structure. For example, left atrium anatomy may be represented by 1000 to 5000 nodes, combined right atrium and right ventricle anatomy may be represented by 50,000 to 100,000 nodes, and / or whole heart anatomy may be represented by 50,000 to 100,000 or more nodes. Before, during, and / or after node correspondence, the 3D models can be transformed (e.g., rotated, translated, warped, and / or scaled) such that respective nodes and / or other features of the 3D models are in similar and / or equivalent relative positions and orientations. Optionally, the 3D models can additionally or alternatively be transformed by the location of common edges, vertices, centroids, anchor points, fiducial markers, model points, markers, etc. An illustrative example of node correspondence is further described below with reference to FIGS. 5 A and 5B.

[0037] After the first plurality of 3D models have been transformed, the SSM can compute a mean shape of the first plurality of 3D models. Computing the mean shape can include performing generalized procrustes analysis (GPA). GPA can include determining an initial estimate for a mean 3D model of the first plurality of 3D models, aligning the remaining 3D models to the mean 3D model, re-determining the estimate of the mean 3D model from thealigned 3D models, and repeating as necessary until the mean 3D model has converged (e.g., does not change significantly upon further iteration).[00381 The SSM can be further configured to determine variation among the first plurality of 3D models in the training data. For example, the SSM can include performing a dimensionality reduction on the first plurality of 3D models. In some variations, the dimensionality reduction is performed using principal component analysis (PCA). In PC A, the first plurality of 3D models are linearly transformed (“mapped”) onto a new coordinate system in which the directions of variations amongst the first plurality of 3D models can be identified. These directions of variations are also known as the principal modes of variation. For example, if the first plurality of 3D models are representative of cardiac anatomy, the SSM can determine that the main modes of variation are the overall size of the heart, width of the heart, size of the chambers, etc. The principal modes of variation can be used to create additional 3D models of the first plurality of 3D models. For example, appropriate values along the principal modes of variation can be used to generate the additional 3D models. The additional 3D models can be generated by modifying any one of the principal modes of variations or multiple principal modes of variations simultaneously.

[0039] Additionally or alternatively, calculation of the SSM can include comparing the first plurality of 3D models to a pre-determined template model. The template model can be stored in a memory of the imaging system and / or anatomical modeling system, and / or received by a network. Based on the template model, each of the first plurality of 3D models can be registered and scaled, e.g., in a similar fashion as the transformation described above. Applying the SSM can include warping portions of the template model onto each of the first plurality of 3D models to produce warped parts. The warped parts can then be used to calculate the mean 3D model and variation as discussed above.

[0040] After the SSM has been applied to the first plurality of 3D models, the first plurality of 3D models can be encoded into a latent space that includes latent variables characterizing the anatomy of interest. Many approaches to encoding the first plurality of 3D models into a latent space can be taken, depending on the machine learning algorithm applied. In some variations, the first plurality of 3D models are encoded into an internal representation or feature vector. The internal representation or feature vector can be embeddings in various ‘hidden layers’ of a neural network, for example. In some variations, the latent space has a lower number of dimensions than the first plurality of 3D models. Thereby, the first pluralityof 3D models can be considered to be “high-dimensional,” and the latent space can be considered to be “low-dimensional.”[00411 In some variations, the latent space includes a plurality of latent variables that represent low-dimensional features of interest of the 3D models. For example, a first latent variable can be related to the size of the 3D model, a second latent variable can be related to the overall geometry of the 3D model, a third latent variable can be related to the curvature of an aspect of the 3D model, and so on. Generally, 3D models among the first plurality of 3D models that are similar (e.g., geometrically, functionally) can be located closer together in the latent space than 3D models that are dissimilar, as a function of similar latent variable values. The latent variables are dynamic and can be retrained, as further discussed below.[0042[ The method 200 can further include decoding the latent space to reconstruct a second plurality of 3D models of the anatomy of interest 206. In some variations, decoding the latent space includes interpolating and / or extrapolating the second plurality of 3D models from the latent space. The decoding can include providing estimates that inform the reconstruction of the second plurality of 3D models. As a rudimentary example for illustrative purposes, a 3D model of a heart from the first plurality of 3D models may be associated with a small value in a latent variable related to size of the heart. Based on the small value, the corresponding 3D model from the second plurality of 3D models may include a reconstruction having a small heart. In some variations, the second plurality of 3D models are reconstructed from the latent space by considering only the latent variables (e.g., without having any other information about the first plurality of 3D models).

[0043] The method 200 can further include determining a reconstruction loss between the second plurality of 3D models and the first plurality of 3D models 208. In general terms, the reconstruction loss represents the degree of dissimilarity between the first plurality of 3D models and the second plurality of 3D models generated using the latent space. The lower the reconstruction loss, the more accurate the machine learning algorithm is at predicting the anatomy of interest from the latent space. The reconstruction loss can be determined at least in part by a loss function, such as mean squared error, mean absolute error, smooth mean absolute error, triplet loss, contrastive loss, quadratic loss, LI loss, L2 loss, Huber loss, quantile loss, cross-entropy loss, log loss, hinge less, or a suitable combination thereof. In some variations, determining the reconstruction loss can include comparing each of the first plurality of 3D models with a respective one of the second plurality of 3D models. The comparison mayinclude a point-to-point, voxel -to-voxel, edge-to-edge, etc. comparison. The reconstruction loss can be determined as an aggregated loss, average loss, or maximum loss.[00441 In some variations, at least one of the first plurality of 3D models can have a greater importance in the determination of the reconstruction loss than the remaining 3D models of the first plurality of 3D models. For example, in training the machine learning algorithm, a user can select a representative 3D model from the first plurality of 3D models and assign importance to that representative 3D model. For example, the representative 3D model may be associated with an increased weight in the reconstruction loss determination. The representative 3D model may be compared with the corresponding one of the second plurality of 3D models and have a multiplied effect, e.g., a loss multiplier of 1.1, 1.3, 1.5, 2, 3, 4, 5, 10, etc. In some variations, the user selects the representative 3D model before inputting the first plurality of 3D models into the machine learning algorithm.[00451 The method 200 can further include modifying at least one of the latent variables until the reconstruction loss satisfies a loss condition 210. Modifying at least one of the latent variables can include changing the value of the latent variable, tuning weights and biases of the machine learning algorithm, changing internal parameters, changing hyperparameters, and more. The selected latent variable to be modified may depend on the machine learning algorithm used. In some variations, the selected latent variable is modified via backpropagation. After the latent variable is changed, the method 200 can proceed with reconstructing the second plurality of 3D models 206 and evaluating the reconstruction loss 208 until the loss condition is satisfied 210.

[0046] In variations where the reconstruction loss is determined at least in part by a loss function, the loss condition can be an internal parameter of the loss function. For example, the loss condition can be defined by a threshold (e.g., the loss condition can be satisfied when the reconstruction loss is less than 30%, 20%, 10%, 5%, 1%, 0.5%, 0.1%, etc.). Additionally or alternatively, the loss condition can include a mathematical optimization. For instance, the loss condition may be an optimization function such that the loss condition is not satisfied until the reconstruction loss hits a local and / or global minimum. The optimization function can be determined by the machine learning algorithm, the user, and / or the manufacturer. In some variations, the optimization function is dynamic, with changing conditions based on the number of allowable iterations. For example, the loss condition can include a maximum number of training iterations (e.g., no more than 5, 10, 20, 50, 100, 500, 1000, 5000, 10,000, or 50,000 iterations).

[0047] In some variations, the machine learning algorithm can be implemented using a train-test split, k-fold cross-validation, and / or other forms of data partitioning. For example, the processes of the method 200 can be applied using only a first subset of the first plurality of 3D models (“training set”), such that the machine learning algorithm learns the latent space representation solely from the training set. Thereafter, the machine learning algorithm can be tested on a second subset of the first plurality of 3D models (“test set”). Reconstruction losses calculated from evaluation of the test set can be used to tune parameters of the machine learning algorithm. In some variations, the tunable parameters include the number of layers, number of ‘neurons’ per layer, the type of cell, output dropout, state dropout, variational dropout, learning rate, decay factor, beta coefficient, and / or maximum number of iterations, etc. The user can tune parameters of the machine learning algorithm until the reconstruction loss is satisfactory or minimized for the training set, the test set, or both. Optionally, the user can additionally partition a third subset of the first plurality of 3D models (“validation set”), with similar applications as the test set.

[0048] As described above, in some variations, a machine learning algorithm of an anatomical modeling system can be trained using complete or full 3D models, incomplete or partial 3D models, or a combination thereof. For example, turning now to FIG. 3, a method 300 for training a machine learning algorithm can incorporate partial 3D models of the anatomy of interest as training data. Similar to that described above with respect to method 200, in some variations the training data may include other information characteristic of anatomy of interest among the training subjects and / or information relating to characteristics of training subjects providing the partial 3D models. In some variations, the method 300 can continue from any of the processes of the method 200, such as to leverage the training data of full 3D models of the anatomy of interest used to train a first machine learning algorithm as described above with respect to FIG. 2. In some variations, some or all of the processes of the method 300 are implemented as the result of computer-readable instructions (e.g., program code) that are configured to be executed by one or more processors (e.g., processor 102a).

[0049] The method 300 can include analyzing a third plurality of 3D models to encode a second latent space of the third plurality of 3D models, where the second latent space comprises a second plurality of latent variables. The third plurality of 3D models can be derived from the first plurality of 3D models of the method 200. For example, each of the third plurality of 3D models can be a truncated and / or sparsified (e.g., downsampled, compressed, or otherwise subsampled) version of a corresponding model from the first plurality of 3D models,such that the third plurality of 3D models are partial versions of the first plurality of 3D models described above with respect to method 200 (that is, the third plurality of 3D models can be reconstructions of various different portions of the anatomy of interest). Where an SSM has been applied to the first plurality of 3D models to generate training data for the machine learning algorithm as described above with respect to FIG. 2, the third plurality of 3D models can be derived from the outputs of the SSM. For instance, each of the third plurality of 3D models can be a truncated version of a corresponding model of the SSM. Encoding the third plurality of 3D models into the second latent space can be generally similar to any of the encoding processes described herein (including, for example, that described above with respect to FIG. 2). In some variations, the third plurality of 3D models are encoded into an internal representation or feature vector having a lower number of dimensions than the third plurality of 3D models.

[0050] The method 300 can further include decoding the second latent space to reconstruct a fourth plurality of 3D models of the anatomy of interest 304. The fourth plurality of 3D models may include fully reconstructed 3D models that are expanded from the partial 3D models. Decoding the second latent space can be generally similar to any of the decoding processes discussed herein (including, for example, that described above with respect to FIG. 2). For instance, in some variations, decoding the second latent space includes interpolating and / or extrapolating the fourth plurality of 3D models from the second latent space. In some variations, the fourth plurality of 3D models include complementary portions of anatomy (e.g., where the third plurality of 3D models correspond to truncated models of the first plurality of 3D models, the fourth plurality of 3D models can include portions of the first plurality of 3D models not included in the third plurality of 3D models). Alternatively, or in combination, the fourth plurality of 3D models can include both anatomical portions represented in the third plurality of 3D models and the complementary portions of anatomy.

[0051] The method 300 can further include determining a reconstruction loss between the first plurality of 3D models and the fourth plurality of 3D models 306. In some variations, the first plurality of 3D models are representative of full anatomies, and the fourth plurality of 3D models include at least some portions not represented in the third plurality of 3D models. The reconstruction loss can be computed in accordance with processes described herein. In some variations, the reconstruction loss is computed between the first plurality of 3D models and the fourth plurality of 3D models by comparing anatomical portions represented in the fourth plurality of 3D models with corresponding portions of the first plurality of 3D models.In some variations, the reconstruction loss is computed by comparing the entirety of each of the first plurality of 3D models with the entirety of each of the fourth plurality of 3D models.[00521 The method 300 can further include modifying at least one of the second latent variables until the reconstruction loss satisfies a second loss condition 308. The second loss condition can be any of the loss conditions described herein. For instance, in some variations, the reconstruction loss is minimized between the first plurality of 3D models and the fourth plurality of 3D models. In some variations, the second loss condition 308 is based on the first latent space. For instance, the second loss condition 308 can be computed between the first latent space and the second latent space. In some variations, the second latent space can be trained until the second latent space approximates and / or is equivalent to the first latent space. Training the second latent space to approximate the first latent space can provide the advantage of increasing the types of inputs the anatomical modeling system can take.

[0053] FIG. 4 is a schematic diagram illustrating a machine learning architecture 400 (“architecture 400”) of an anatomical modeling system (e.g., anatomical modeling system 102). The architecture 400 can be used for evaluating sparse medical imaging data, in accordance with variations of the present technology. The architecture 400 can be constructed using the method 200 of FIG. 2, or include a combination of the processes of the method 200 and those further described below. The architecture 400 can be implemented across any desired software and / or hardware components by any of the systems and devices described herein, such as the anatomical modeling system 102 and / or imaging system 104 of FIG. 1.

[0054] In some variations, the architecture 400 includes a machine learning algorithm 402 configured to receive input data 404 and produce model outputs 406. The architecture 400, once trained, can be configured to have learned a latent space representation of an anatomy of interest and configured to predict unknown portion(s) of the anatomy of interest given known portions of the anatomy of interest. The machine learning algorithm 402 is depicted as an encoder-decoder network for illustrative purposes only. Many suitable types of networks and machine learning algorithms can be used in accordance with variations of the present technology. For example, the machine learning algorithm 402 can include convolutional neural networks (CNNs) having convolutional layers, pooling layers, and / or fully-connected layers, variational autoencoders, generative adversarial networks (GANs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), Siamese neural networks, vision transformers, regression algorithms, and dimensionality reduction algorithms.

[0055] The input data 404 can include a first plurality of 3D models of an anatomy of interest. The first plurality of 3D models can represent medical imaging reconstructions of the anatomy of interest from a population of subjects. In some variations, the first plurality of 3D models are representative of hearts reconstructed from the population of subjects. The architecture 400 can include analyzing the first plurality of 3D models using a statistical shape model (SSM). The SSM can be constructed in a similar fashion to the SSMs described elsewhere herein, and / or as described below with reference to FIGS. 5A and 5B.

[0056] FIGS. 5 A and 5B are schematic diagrams relating to an example SSM 400, in accordance with variations of the present technology. FIG. 5A depicts a node correspondence process of the SSM 500. The node correspondence process includes comparing an original 3D model 502 with a template model 504 and producing a warped model 506, the warped model 506 being a transformed representation of the original 3D model 502. The original 3D model 502 can be selected from a plurality of 3D models (e.g., the first plurality of 3D models of FIG. 5). The original 3D model 502 can be a representation of a subject’s heart in either an unmodified or pre-processed configuration. The original 3D model 502 can be initially oriented and / or positioned in a variety of ways. For example, while the original 3D model 502 is depicted as having a substantially similar orientation to template model 504, the original 3D model 502 may instead be misaligned, rotated, translated, inverted, etc.

[0057] The template model 504 can be a pre-determined 3D model stored in a memory of the anatomical modeling system. The template model 504 can be arranged in a specific orientation and position suitable for processing. The template model 504 can be determined by modeling software and / or learned from a repository of medical imaging data. A first set of one or more nodes 508 can be located on the template model 504. For example, each node of the first set of nodes 508 can correspond to an anatomical feature of interest such as a specific feature of the right ventricle. The first set of nodes 508 can include at least 1 node, 10 nodes, 100 nodes, 1000 nodes, etc. The first set of nodes 508 can be automatically identified using segmentation and labeling algorithms or manually by a user of the anatomical modeling system.

[0058] Comparing the original 3D model 502 with the template model 504 can include identifying a second set of one or more nodes of the original 3D model 502, where the second set of nodes spatially correspond to the first set of nodes. In other words, corresponding nodes of the original 3D model 502 and the template model 504 can represent a similar anatomical feature (e.g., specific feature of the left ventricle). Based on the comparison, the warped model506 can be produced. In some variations, the warped model 506 can represent a transformation of the original 3D model 502 to match the orientation, position, and / or size of the template model 504. For instance, the transformation can include one or more of aligning, scaling, rotating, inverting, or translating the original 3D model 502 based on the template model 504. The warped model 506 can represent the original 3D model 502 in a configuration suitable for processing by the SSM.[00591 Each of the first plurality of 3D models can be treated like the original 3D model502 as described above, thereby producing a plurality of warped models based on training data. The SSM can then be computed from the plurality of warped models. In some embodiments, the SSM is computed by applying statistical methods to the warped models. For instance, the SSM can be computed by applying PCA to the warped models. The outputs of the SSM can include the plurality of warped models, descriptive statistics of the variations amongst the warped models, and an average model 510 (depicted in FIG. 5B). The average model 510 can be representative of the average anatomical features across all 3D models of the plurality of 3D models.[0060 J Returning now to FIG. 4, after the SSM is constructed, outputs from the SSM (“model inputs”) are input into the machine learning algorithm 402. In some variations, the machine learning algorithm 402 includes an encoder 408 and a decoder 410. The encoder 408 can be configured to encode a latent space representation of the model inputs. In some variations the encoding is a multi-layer process. For example, the encoding can include a first layer 412. The first layer 412 can be configured to receive the model inputs and perform a transformation of the model inputs. The first layer 412 can include a lower-dimensional representation of the model inputs. The first layer 412 can be the first amongst a plurality of encoding layers 412. The plurality of encoding layers can be configured to compress data from a preceding layer to a succeeding layer of the plurality of encoding layers. The compression can include, for example, a flattening of the data.[00611 The first layer 412 can be configured to compress the model inputs into an embedding 414. The embedding 414 can include the latent space representation of the anatomy of interest. In some variations, the embedding 414 includes a plurality of latent variables. Each of the plurality of latent variables can be related to a feature of the anatomy of interest. For example, a first latent variable of the plurality of latent variables can be related to an overall size of the anatomy of interest and a second latent variable of the plurality of latent variablescan be related to an overall geometric profile of the anatomy of interest. In some variations, the architecture 400 stores the latent variables for later retrieval.[00621 The decoder 410 can be configured to receive the embedding 414 (e.g., the latent variables) and transform the embedding into a second layer 416. The second layer 416 can represent a higher dimensional representation of the embedding. In some variations, the second layer 416 is the first amongst a plurality of decoding layers 416. The plurality of decoding layers 416 can be configured to decompress data from a preceding layer to a succeeding layer of the plurality of decoding layers 416. The decompression can include, for example, an interpolation and / or extrapolation of the data.

[0063] The decoder 410 can be configured to transform the second layer 416 into the model outputs 406. In some variations, transforming the second layer 416 into the model outputs 406 includes generating the model outputs 406 using the one or more latent variables. In some variations, the decoder 410 has no information about the model inputs 404 other than the embedding 414 and the second layer 416.

[0064] After the model outputs 406 have been produced, the machine learning algorithm 402 can compare the model outputs 406 with the model inputs 404. The comparison can include computing a reconstruction loss. While the model outputs 406 and the model inputs 404 can be compared directly, the reconstruction loss can additionally or alternatively be computed at any level of the architecture 400. In some variations, the reconstruction loss is a reflection of the mismatch between the model outputs 406 and the model inputs. The reconstruction loss can be determined by a loss function, such as mean squared error, mean absolute error, smooth mean absolute error, triplet loss, contrastive loss, quadratic loss, LI loss, L2 loss, Huber loss, quantile loss, cross-entropy loss, log loss, hinge less, or a suitable combination thereof. In some variations, determining the reconstruction loss can include comparing each of the model inputs 404 with a respective one of the model outputs 406. The comparison may include a point-to-point, voxel -to- voxel, edge-to-edge, etc. comparison. The reconstruction loss can be determined as an aggregated loss, average loss, or maximum loss.

[0065] Based on the reconstruction loss, the machine learning algorithm 402 can retrain and / or modify one or more of the first layer 412, the embedding 414, and the second layer 416. For example, the machine learning algorithm 402 can perform b ackpropagation to reduce the reconstruction loss. Backpropagation can include tuning the operations of one or more of the first layer 412 and the second layer 416. Backpropagation can further include modifying theembedding 414. For example, one or more latent variables of the embedding 414 can be altered. In some variations, backpropagation is performed iteratively until the reconstruction loss satisfies a loss condition. For example, the machine learning algorithm 402 can terminate the training process when the reconstruction loss is below an acceptable error tolerance. In other variations, the machine learning algorithm 402 can terminate the training process when the reconstruction loss meets a mathematical determination (e.g., a minimization). Optionally, the machine learning algorithm 402 can terminate the training process after a pre-determined number of iterations (e.g., 10,000 iterations).[0066| The machine learning algorithm 402 can be implemented using a train-test split, k-fold cross-validation, and other forms of data partitioning. For example, the architecture 400 can be applied using only a first subset of the model inputs (“training set”), such that the machine learning algorithm 402 learns the embedding 414 solely from the training set. Thereafter, the machine learning algorithm 404 can be tested on a second subset of the model inputs (“test set”). Reconstruction losses calculated from evaluation of the test set can be used to tune parameters of the machine learning algorithm 404. In some variations, the tunable parameters include the number of layers, number of ‘neurons’ per layer, the type of cell, output dropout, state dropout, variational dropout, learning rate, decay factor, beta coefficient, maximum number of iterations, etc. The user can tune parameters of the machine learning algorithm 404 until the reconstruction loss is satisfactory or minimized for the training set, the test set, or both. Optionally, the user can additionally partition a third subset of the model inputs (“validation set”), with similar applications as the test set.[00671 FIG. 9 is a schematic diagram illustrating a machine learning architecture 900 (“architecture 900”) of an anatomical modeling system such as anatomical modeling system 102, in accordance with variations of the present technology. The architecture 900 can be utilized using the method 300 of FIG. 3 or a combination of the processes of the method 300 and those further described herein. In some variations, the architecture 900 includes the architecture 400 or a suitable replacement thereof. The architecture 900 can be implemented across any desired software and / or hardware component by any of the systems and devices described herein, such as the anatomical modeling system 102 and / or imaging system 104 of FIG. 1.

[0068] In some variations, the architecture 900 can receive a first plurality of 3D models 902. The first plurality of 3D models can be generally similar to the first plurality of 3D models described with respect to the method 200 of FIG. 2. The first plurality of 3D modelscan be processed by an SSM 904. The SSM 904 can, for example, identify variations across the first plurality of 3D models and generate additional models of the first plurality of 3D models.

[0069] After the SSM has been applied to the first plurality of 3D models, the architecture 900 can include processing the first plurality of 3D models in whole and / or in part, as discussed in further detail below. In some variations, the first plurality of 3D models can be processed at least in part in a manner similar to that described above with respect to method 300 and FIG. 3.

[0070] In a first aspect, processing the first plurality of 3D models includes evaluating the entirety of each of the first plurality of 3D models (“full 3D models”) 906, where each of the full 3D models is a representation of the full anatomy of interest (e.g., full heart). For instance, the full 3D models can be used to encode a latent space of a first machine learning algorithm 908. The full 3D models can be decoded to generate a second plurality of 3D models 910, and the latent space can be updated based on a loss condition, as described above.

[0071] In a second aspect, processing the first plurality of 3D models includes evaluating limited portions of each of the first plurality of 3D models (“partial 3D models”) 912, where each of the partial 3D models is a representation of a portion of the anatomy of interest (e.g., each partial 3D can be a random subsampled portion of a full 3D model). For instance, the partial 3D models can be used to encode a second latent space of a second machine learning algorithm 914. The second latent space can be decoded to generate a plurality of expanded 3D models 916. In some variations, a reconstruction loss can be computed between the expanded 3D models 916 and the full 3D models 906. Based on the reconstruction loss, the second latent space can be modified.

[0072] After the first machine learning algorithm 908 and / or the second machine learning algorithm 914 have been trained using the respective full 3D models 906 and partial 3D models 914, the architecture 900 can be configured to receive an incomplete 3D model to generate an expanded 3D model of an anatomy of interest, as will be discussed further below. In some variations, the first machine learning algorithm 908 and the second machine learning algorithm 914 can be part of the same algorithm, and the full 3D models and the partial 3D models can be inputted simultaneously.B. USE OF AN ANATOMICAL MODELING SYSTEM

[0073] FIG. 6 is a flow chart illustrating a method 600 for anatomical modeling with a machine learning algorithm, in accordance with variations of the present technology. The method 600 can be performed using any of the systems and devices described herein, such as the anatomical modeling system 102 of FIG. 1. The machine learning algorithm can be trained as described above with reference to FIGS. 2 and 3, as described below, or by a combination thereof. In some variations, some or all of the processes of the method 600 are implemented as the result of computer-readable instructions (e.g., program code) that are configured to be executed by one or more processors (e.g., processor 102a of the anatomical modeling system 102 of FIG. 1).

[0074] The method 600 can include receiving an incomplete three-dimensional (3D) model of an anatomy of interest of a subject 602. The incomplete 3D model can include a partially-reconstructed model of the anatomy of interest derived from medical imaging data (e.g., reconstructing only a portion of the anatomy of interest, such as due to sparse imaging data). Optionally, the method 600 can include receiving the medical imaging data, the method 600 further including generating the partially-reconstructed model from the medical imaging data. In some variations, the medical imaging data is collected using computed tomography. The medical imaging data can include sparse imaging data. For example, the medical imaging data may have missing values corresponding to unknown portions of the anatomy of interest. Alternatively, or in combination, the medical imaging data may include one or more of noise, signal artefacts, incorrect values, low-confidence values, annotations, or corrupted data. The incomplete 3D model can be reconstructed using an imaging system or suitable computing system.

[0075] In some variations, the method 600 may include receiving other information characteristic of anatomy of interest of a subject (e.g., dimensional measurements of the anatomy of interest). For example, in variations in which the anatomy of interest is or includes a heart, the training data may include dimensional measurements of anatomy around the heart (e.g., chest circumference) and / or the heart itself (e.g., right ventricle axis length, left ventricle axis length, volume and / or width of one or more heart chambers (left atrium, right atrium, left ventricle, right ventricle, etc.), diameter or other dimension(s) of one or more heart valves (aortic valve, mitral valve, etc.), and / or other suitable anatomical measurements alone or in combination providing numerical characterization of the anatomy of interest. Such information may, for example, be received in addition to the incomplete 3D model, such that theinformation and the incomplete 3D model may be analyzed by a pre-trained machine learning algorithm as described below.

[0076] Additionally or alternatively, in some variations, the method 600 may include receiving information relating to one or more characteristics of the subject. Such information may include but is not limited to sex, age, weight, height, BMI, lifestyle, and / or medical condition (e.g., disease state), etc. of the subject. Such information may, for example, be received in addition to the incomplete 3D model, such that the information and the incomplete 3D model may be analyzed by a pre-trained machine learning algorithm as described below.

[0077] The method 600 can further include inputting the incomplete 3D model (and / or information characteristic of the anatomy of interest, such as dimensional measurements, and / or information relating to one or more characteristics of the subject) into a pre-trained machine learning algorithm 604, wherein the machine learning algorithm encodes a latent space representation of the anatomy of interest from training data derived from a plurality of training subjects 604. In some variations, the incomplete 3D model and / or other information described above is automatically inputted into the pre-trained machine learning algorithm. For example, the incomplete 3D model may be collected by an imaging system coupled, directly or indirectly via a network, to an anatomical modeling system comprising the pre-trained machine learning algorithm. Additionally or alternatively, dimensional measurements of the anatomy of interest (e.g., based on the incomplete 3D model and / or from other suitable techniques) may be automatically or manually inputted into the pre-trained machine learning algorithm. Additionally or alternatively, information relating to one or more characteristics of the subject may be automatically inputted into the pre-trained machine learning algorithm via an electronic health record system, and / or manually inputted into the pre-trained machine learning algorithm.

[0078] In some variations, the machine learning algorithm includes a machine learning algorithm having an architecture equivalent or similar to the architecture 400 of FIG. 4 and / or FIG. 9. For example, the machine learning algorithm can include an encoder-decoder network having a plurality of hidden layers. In some variations, the machine learning algorithm can include a convolutional neural network (CNN) having convolutional layers, pooling layers, and / or fully-connected layers. Other machine learning algorithms suitable for performing the processes of the present technology include, but are not limited to, variational autoencoders, generative adversarial networks (GANs), recurrent neural networks (RNNs), long short-termmemory networks (LSTMs), Siamese neural networks, vision transformers, regression algorithms, and dimensionality reduction algorithms.[00791 The machine learning algorithm can be trained using any of the techniques described herein, such as the method 200 of FIG. 2. For example, the machine learning algorithm can be trained from a database of input models of subject anatomy from a plurality of training subjects. The database of input models can be processed by a statistical shape model (SSM). For example, the SSM can be configured to determine the principal modes of variation amongst the input models and generate additional models. Output data from the SSM (hereafter referred to as “training data”) can be fed into the machine learning algorithm. In some variations, the training data includes one or more of the input models and additional models generated by the SSM. The training data can also include one or more of the input models and / or additional models in an aligned configuration, feature maps of the variation across the input and / or additional models, and / or a mean model of the input and / or additional models.[0080| Similar to that described above, the machine learning algorithm can be configured to encode a latent space representation (e.g., embedding) of the anatomy of interest from the training data. The latent space representation can include a data compression from the training data. In some variations, the latent space representation has a plurality of latent variables. Each of the latent variables can be related to an aspect of the training data. For example, a first latent variable of the latent variables can be related to the height of the anatomy of interest, a second latent variable of the latent variables can be related to the width of the anatomy of interest, and a third latent variable of the latent variables can be related to the depth of the anatomy of interest.

[0081] The machine learning algorithm can be configured to decode the latent space representation of the anatomy of interest to generate the output models. Decoding the latent space representation can include interpolating, extrapolating, or otherwise decompressing the latent space representation. In some variations, the decoding occurs without direct knowledge of the input models. Decoding can occur in multiple layers, as previously explained with reference to the architecture 400 of FIG. 4.

[0082] The output models can then be compared to the input models. Comparing the output models to the input models can include evaluating a reconstruction loss between corresponding pairs of input and output models. The machine learning algorithm can be re-trained as necessary until the reconstruction loss is sufficiently low. This determination can be based on an optimization algorithm, a user input, pre-set threshold, etc.

[0083] Inputting the incomplete 3D model into the pre-trained machine learning algorithm 604 can include pre-processing the incomplete 3D model into a data format compatible with the machine learning algorithm. For example, the pre-processing may include one or more of converting between file formats, aligning the incomplete 3D model in accordance with the SSM of the pre-trained machine learning algorithm, or applying a corrective algorithm (e.g., contrast enhancement). As previously noted, the incomplete 3D model may include only a partial representation of the anatomy of interest. Accordingly, alignment of the incomplete 3D model may include performing a node correspondence between the incomplete 3D model and the input models of the SSM. The node correspondence can be generally similar to other correspondence techniques discussed herein.

[0084] The method 600 can further include generating, using the machine learning algorithm, an expanded 3D model of the anatomy of interest of the subject 606. For example, the expanded 3D model can include a reconstruction of one or more portions of the anatomy of interest that are not reconstructed in the incomplete 3D model (e.g., where the incomplete 3D model includes only a reconstruction of a first portion of the anatomy of interest, the expanded 3D model can include a reconstruction of at least a second portion of the anatomy of interest that is different from a first portion of the anatomy of interest). In some variations, the incomplete 3D model is treated the same as each of the input models during the training of the machine learning algorithm. For example, the incomplete 3D model may be processed, encoded into a latent space representation, and decoded from the latent space representation into the expanded 3D model.

[0085] In some variations, generating the expanded 3D model includes comparing the latent space representation of the incomplete 3D model with any previous latent space representations learned using the machine learning algorithm. For example, the machine learning algorithm may identify 3D models of the database of input models that are similar to the incomplete 3D model via their relative positions in the latent space. 3D models having closer proximity in the latent space may have increased resemblance (e.g., geometry, size) compared to 3D models farther from one another in the latent space. Accordingly, the machine learning algorithm can generate the expanded 3D model by determining latent variables of the latent space representation of the incomplete 3D model and decoding the latent variables.

[0086] In some variations, the incomplete 3D model comprises a first model portion corresponding to a first anatomical region and generating the expanded 3D model includes outputting, from the machine learning algorithm, a second model portion corresponding to a second anatomical region. In such cases, the method 600 can include generating the expanded 3D model of the anatomy of interest of the subject by combining the incomplete 3D model and the second model portion. In some variations, combining the incomplete 3D model and second model portion includes adding the second model portion to the incomplete 3D model, or vice- versa. Further corrections may be required such as adjoining edges, resolving intersecting portions, and / or filling in missing values. The expanded 3D model can then be communicated to one or more of a user, the subject, or a medical practitioner.

[0087] In some variations, the method 600 includes visualizing the expanded 3D model on a display 608. The display can be an LCD or LED screen, VR / AR visualization, etc. Further example details of the visualizing are described below with reference to FIG. 8. In some variations, multiple iterations of inputting the incomplete 3D model into the pre-trained machine learning algorithm may be useful. For example, a user can repeat any of the processes of method 600 to identify possible alternative anatomies and improve information gain.

[0088] FIG. 7 is a schematic diagram illustrating a machine learning architecture 700 (“architecture 700”) of an anatomical modeling system, in accordance with variations of the present technology. The architecture 700 can be utilized using the method 600 of FIG. 6 or a combination of the processes of the method 600 and those further described herein. In some variations, the architecture 700 includes the architecture 400 or a suitable replacement thereof. The architecture 700 can be implemented across any desired software and / or hardware component by any of the systems and devices described herein, such as the anatomical modeling system 102 and / or imaging system 104 of FIG. 1.

[0089] In some variations, the architecture 700 includes a machine learning algorithm 702. The machine learning algorithm 702, once trained, can be configured to receive an incomplete 3D model of a first portion of an anatomy of interest of a subject and generate an expanded 3D model of the anatomy of interest of the subject 710. The machine learning algorithm 702 can include supervised learning models, unsupervised learning models, semisupervised learning models, and / or reinforcement learning models. Examples of machine learning models suitable for use with the present technology include, but are not limited to: regression algorithms, instance-based algorithms, regularization algorithms, decision tree algorithms, Bayesian algorithms, clustering algorithms, association rule learning algorithms,artificial neural networks, deep learning algorithms (e.g., convolutional neural networks, recurrent neural networks, long short-term memory networks, stacked autoencoders, Siamese neural networks, generative adversarial networks, capsule networks, graph neural networks, encoder-decoder networks, deep Boltzmann machines, deep belief networks, transformers, vision transformers, language models), principal component analysis (PCA), singular value decomposition (SVD), other dimensionality reduction algorithms, time series forecasting algorithms, and ensemble algorithms.[00901 In some variations, the machine learning algorithm 702 is trained using a first plurality of 3D models 704 as processed by a statistical shape model 706. The first plurality of 3D models 704 can be in a stored repository of 3D models of the anatomy of interest. For example, the first plurality of 3D models 704 can include 3D models of hearts. The first plurality of 3D models 704 are processed by the SSM 706 in accordance with the processes described herein. For example, the first plurality of 3D models 704 can undergo pre-processing and node correspondence. In some variations, the first plurality of 3D models are transformed into a plurality of warped 3D models. The warped 3D models can be used to compute a SSM describing the principle modes of variation amongst the first plurality of 3D models, in addition to an average 3D model. The SSM can also be used to generate additional 3D models of the first plurality of 3D models 704.

[0091] The SSM, and / or derivatives thereof, can be used to train the machine learning algorithm 702. In some variations, the first plurality of 3D models 704, the SSM 706, and the machine learning algorithm 702, are replaceable by the architecture 400. For example, the architecture 400 can be trained independently and incorporated into the architecture 700 for later use such that the user does not need to retrain the machine learning algorithm 702.

[0092] Once the machine learning algorithm 702 has been trained, the machine learning algorithm 702 can be configured to receive an incomplete 3D model 708 of a first portion of an anatomy of interest. The incomplete 3D model 708 can be received separately from the first plurality of 3D models 704. In some variations, the incomplete 3D model 708 can be a partial reconstruction of the anatomy of interest. For example, the incomplete 3D model 708 can be reconstructed from sparse medical imaging data collected using ultrasound. In some variations, the incomplete 3D model 708 includes both known portions of anatomy and unknown portions of anatomy. The unknown portions of anatomy can include missing values, artefacts, and / or low-confidence data.

[0093] The machine learning algorithm 702 can process the incomplete 3D model 708 as model input into the machine learning algorithm 702 and generate an expanded 3D model of the anatomy of interest. The expanded 3D model 710 can include portions of anatomy not represented in the incomplete 3D model 708. In some variations, generating the expanded 3D model 710 includes comparing the incomplete 3D model 708, or derivatives thereof, with a latent space representation of the anatomy of interest, wherein the machine learning algorithm comprises the latent space representation. For example, the incomplete 3D model 708 may be processed in a similar way to the first plurality of 3D models 704 and decoded to generate the expanded 3D model.

[0094] In some variations, the incomplete 3D model 708 comprises a first model portion corresponding to a first anatomical region and generating the expanded 3D model 710 includes outputting, from the machine learning algorithm 702, a second model portion corresponding to a second anatomical region. For instance, the incomplete 3D model 708 and the second model portion can be combined to generate the expanded 3D model 710 including both the incomplete 3D model 708 (which is known from imaging data) and the second model portion (which is predicted and generated based on the pre-trained machine learning algorithm 702). The expanded 3D model functions as a fully-reconstructed 3D model. In some variations, the expanded 3D model 710 is visualized, as discussed below for example with reference to FIG. 7. In some variations, the expanded 3D model 710 can be stored in the anatomical modeling system and / or other suitable memory device. In some variations, the expanded 3D model 710 can be stored in an electronic medical record (EMR) of the subject and / or communicated to any suitable user. For example, the expanded 3D model 710 can be communicated and used to inform one or more of a medical practitioner, the subject, and / or other user.

[0095] In some variations, the expanded 3D model, also known as the fully- reconstructed 3D model, can be sent to a display. For example, as shown in FIG. 8, a fully- reconstructed 3D model 802 can be shown on a display 804. The fully-reconstructed 3D model 802 can include original (e.g., known) portions 806 and predicted portions 808 combined together. The display 804 can be an LED monitor, LCD screen, virtual reality display, projection, AR / VR visualization, smart glasses, etc. The display 804 can be located in an operating room or other point-of-care location, and communicatively coupled to the anatomical modeling system. In some variations, the display 804 is moveable around the operating room. The display 804 need not be in an operating room, however.

[0096] The original portions 806 can correspond to the incomplete 3D model of the first portion of the anatomy of interest. Additionally or alternatively, the original portions 806 can correspond to other imaging data (e.g., previous scans). The original portions 806 can be intended to indicate known portions of the subject’s anatomy. In some variations, the original portions 806 cannot be modified by a user of the anatomical modeling system.

[0097] The predicted portions 808 can correspond to the predicted second model portions of the anatomy of interest. Additionally or alternatively, the predicted portions 808 can correspond to other imaging data (e.g., edited scans). The predicted portions 808 can be configured to communicate unknown portions of the subject’s anatomy. In some variations, the anatomical modeling system allows a user to modify the predicted portions 808 how the predicted portions 808 are displayed on the display 804. Alternatively, or in addition, the predicted portions 808 can be a first predicted section and the user can generate additional predicted portions for display. The additional predicted portions may be derived from outputs from re-running the machine learning algorithm. The anatomical modeling system can be configured to allow a user to select desired predicted portions from a plurality of predicted portions 808.

[0098] In some variations, the original portions 806 and the predicted portions 808 of the anatomy of interest can be displayed in different manners, such that a user can distinguish between portions of the reconstruction that are based on original imaging data, and portions of the reconstruction that are predicted using methods and systems described herein (e.g., with a trained machine learning algorithm). For example, in some variations, the original portions 806 have a first shading and the predicted portions 808 have a second shading different from the first shading. Alternatively, or in combination, the predicted portions 808 can be differentiated from the original portions 806 by one or more of patterns (e.g., hatching), color, heatmaps, gradients, transparency effects, and / or other appearance modifications. In some variations, the user can selectively display only one of the original portions 806 or the predicted portions 808 (e.g., “toggle” on or off the display of the original portions 806 and / or the display of the predicted portions 808). Optionally, the user can select preferred mode(s) of display, such as customized color palettes, transparency settings, and / or display styles. The display 804 can additionally include renderings not represented in any of the 3D models. For example, the display 804 can incorporate additional surface topographies from medical devices (e.g., a pacemaker). In such instances, the additional surface topology may be represented by a third shading different from the first and second shadings.

[0099] In some variations, the display of the fully-reconstructed 3D model can be manipulated or otherwise controlled by a user in one or more several ways. For example, a user interface device can allow a user to rotate, translate, scale, zoom in, zoom out, etc. the display of the fully-reconstructed 3D model. As another example, the user interface device can allow a user to select a cross-sectional view of the fully-reconstructed 3D model for display (e.g., along a selected or otherwise designated plane). Suitable user interface devices include, for example, a computer mouse, touch screenjoystick, handheld controller, and / or the like.

[0100] The various processes described herein can be partially or fully implemented using program code including instructions executable by one or more processors of a computing system for implementing specific logical functions or steps in the process. The program code can be stored on any type of computer-readable medium, such as a storage device including a disk or hard drive. Computer-readable media containing code, or portions of code, can include any appropriate media known in the art, such as non-transitory computer-readable storage media. Computer-readable media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage and / or transmission of information, including, but not limited to, random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technology; compact disc read-only memory (CD-ROM), digital video disc (DVD), or other optical storage; magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices; solid state drives (SSD) or other solid state storage devices; or any other medium which can be used to store the desired information and which can be accessed by a system device.II. Examples of Imaging Reconstruction

[0101] The image reconstruction devices, methods, and systems described herein can be used for a variety of applications. The following examples are included to further describe some aspects of the present technology and should not be used to limit the scope of the technology.Example: Reconstructing a Subject’s Heart

[0102] An example of an anatomical modeling system configured to reconstruct a patient’s heart while the patient is undergoing a cardiac implant procedure (e.g., placement of a cardiac pacemaker) will now be described. In this example, the anatomical modeling system receives input prior to or during the procedure. The input can be a plurality of medical imagesand / or models. For example, the anatomical modeling system may be provided with a database including 500 3D models of hearts from a variety of training subjects. The 3D models may, for example, be stored in a repository derived from medical imaging of other subjects. The anatomical modeling system may be instructed to learn a latent space representation of anatomy of the heart based on the repository. Learning the latent space representation includes applying a statistical shape model to the 3D models, as described in further detail herein. After the latent space representation is learned, e.g., via a machine learning algorithm, the anatomical modeling system may be ready to receive additional input data for reconstructing a full 3D model based on sparse imaging of a subject’s heart, using the now-trained machine learning algorithm.

[0103] During a cardiac implant procedure for a patient, a user may collect concurrent medical imaging data of the subject. The medical imaging data can be collected using transthoracic echocardiography. In general, the medical imaging data may be sparse, in that it may insufficiently image certain portions of the patient’s heart. For example, the user may collect medical imaging data that adequately captures the lower chambers of the patient’s heart without adequately capturing the upper chambers of the patient’s heart. Insufficient imaging may occur, for example, due to restrictions in the field of view of the ultrasound probe and / or structures (e.g., ribs) that may block views around the heart. The medical imaging data may be transferred to the anatomical modeling system via a network. A partial 3D reconstruction of the patient’s heart based on the medical imaging data may be generated by the anatomical modeling system and / or a third party system. Such a partial 3D reconstruction includes known portions of the patient’s heart in that the partial 3D reconstruction is based on actual medical imaging data obtained. However, the partial 3D reconstruction is missing portions of the patient’s heart that correspond to portions of the patient’s heart not sufficiently imaged.

[0104] The anatomical modeling system inputs the 3D construction of known portions of the patient’s heart (“original portions”) into the pre-trained machine learning algorithm. Generally, the pre-trained machine learning algorithm may compare the original portions of the patient’s heart, or derivations thereof, with the latent space representation and be used to generate an expanded 3D reconstruction including the original portions and predictions for the unknown portions of the patient’s heart (“predicted portions”) The user can view and / or interact with the displayed full 3D construction to improve the cardiac implant procedure. For example, the user can reference the full 3D reconstruction for purposes of navigating the patient’s heart, placing the cardiac implant, etc.

[0105] In some variations, the prediction of the unknown portions of the patient’s heart can be repeated multiple times. For example, the anatomical modeling system can generate predicted portions continuously or intermittently throughout the procedure. The predictions may change based on the original portions of anatomy collected in the medical imaging data. The user can also instruct the anatomical modeling system to store ‘snapshots’ of the anatomy including both original portions and predicted portions throughout the cardiac implant procedure.

[0106] Although in this example the model reconstruction process using the machine learning algorithm is primarily described as being performed intra-procedurally, it should be understood that additionally, or alternatively, it can be performed pre-procedurally (e.g., for treatment planning purposes) and / or post-procedurally (e.g., during follow-up after implant placement).Conclusion

[0107] Although many of the variations are described above with respect to systems, devices, and methods for reconstructing models of subject anatomy from sparse imaging data, the technology is applicable to other applications and / or other approaches, such as reconstructing other image types and models. Moreover, other variations in addition to those described herein are within the scope of the technology. Additionally, several other variations of the technology can have different configurations, components, or procedures than those described herein. A person of ordinary skill in the art, therefore, will accordingly understand that the technology can have other variations with additional elements, or the technology can have other variations without several of the features shown and described above with reference to FIGS. 1-9.

[0108] The descriptions of variations of the technology are not intended to be exhaustive or to limit the technology to the precise form disclosed above. Where the context permits, singular or plural terms may also include the plural or singular term, respectively. Although specific variations of, and examples for, the technology are described above for illustrative purposes, various equivalent modifications are possible within the scope of the technology, as those skilled in the relevant art will recognize. For example, while steps are presented in a given order, alternative variations may perform steps in a different order. The various variations described herein may also be combined to provide further variations.

[0109] As used herein, the terms “generally,” “substantially,” “about,” and similar terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent variations in measured or calculated values that would be recognized by those of ordinary skill in the art.

[0110] Moreover, unless the word “or” is expressly limited to mean only a single item exclusive from the other items in reference to a list of two or more items, then the use of “or” in such a list is to be interpreted as including (a) any single item in the list, (b) all of the items in the list, or (c) any combination of the items in the list. Additionally, the term "comprising" is used throughout to mean including at least the recited feature(s) such that any greater number of the same feature and / or additional types of other features are not precluded. It will also be appreciated that specific variations have been described herein for purposes of illustration, but that various modifications may be made without deviating from the technology. Further, while advantages associated with certain variations of the technology have been described in the context of those variations, other variations may also exhibit such advantages, and not all variations need necessarily exhibit such advantages to fall within the scope of the technology. Accordingly, the disclosure and associated technology can encompass other variations not expressly shown or described herein.

Claims

CLAIMSI / We claim:

1. A method compri sing : receiving an incomplete three-dimensional (3D) model of an anatomy of interest of a subject; inputting the incomplete 3D model into a pre-trained machine learning algorithm, wherein the machine learning algorithm encodes a latent space representation of the anatomy of interest from training data derived from a plurality of training subjects; and generating, using the machine learning algorithm, an expanded 3D model of the anatomy of interest of the subject.

2. The method of claim 1, wherein the expanded 3D model is directly outputted from the machine learning algorithm.

3. The method of claim 1 or 2, wherein the incomplete 3D model comprises a first model portion corresponding to a first anatomical region and generating the expanded 3D model includes outputting, from the machine learning algorithm, a second model portion corresponding to a second anatomical region.

4. The method of any one of claims 1-3, wherein the training data comprises results from applying statistical shape modeling on a plurality of 3D models of the anatomy of interest corresponding to the plurality of training subjects.

5. The method of any one of claims 1-4, wherein the machine learning algorithm comprises an artificial neural network.

6. The method of any one of claims 1-5, wherein the latent space representation includes a lower-dimensional representation of the anatomy of interest.

7. The method of any one of claims 1-6, wherein the latent space representation includes one or more latent variables related to at least one of a size or geometry of the anatomy of interest.

8. The method of any one of claims 1-7, wherein the anatomy of interest is a heart.

9. The method of any one of claims 1-8, wherein the incomplete 3D model is reconstructed from one or more of fluoroscopy, ultrasound, X-ray, computed tomography, positron emission tomography, magnetic resonance imaging, or single-photon emission computed tomography.

10. The method of any one of claims 1-9, further comprising displaying the expanded 3D model on a display.

11. The method of claim 10, wherein displaying the expanded 3D model comprises displaying a first portion of the anatomy of interest in accordance with a first display scheme, and displaying a second portion of the anatomy of interest in accordance with a second display scheme.

12. A system comprising: a processor; and a memory operably coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving, via the processor, an incomplete 3D model of an anatomy of interest of a subject; inputting, via the processor, the incomplete 3D model into a pre-trained machine learning algorithm, wherein the machine learning algorithm encodes a latent space representation of the anatomy of interest from training data derived from a plurality of training subjects; and generating, via the processor, using the machine learning algorithm, an expanded 3D model of the anatomy of interest of the subject.

13. The system of claim 12, wherein the incomplete 3D model comprises a first model portion corresponding to a first anatomical region and generating the expanded 3D model includes outputting, via the processor, from the machine learning algorithm, a second portion corresponding to a second anatomical region.

14. The system of claim 12 or 13, wherein the training data comprises results from applying statistical shape modeling on a plurality of 3D models of the anatomy of interest corresponding to the plurality of training subjects.

15. The system of any one of claims 12-14, wherein the incomplete 3D model is reconstructed from one or more of fluoroscopy, ultrasound, X-ray, computed tomography, positron emission tomography, magnetic resonance imaging, or single-photon emission computed tomography.

Images