3D Anatomical Reconstruction from 2D Images for Non-Rigid Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image-guided radiotherapy technologies are limited by the inability to accurately track non-rigid motion of internal anatomy during free-breathing therapies, such as liver cancer treatments, due to reliance on 2D slice imaging that fails to capture out-of-plane motion, leading to errors in radiation targeting and potential damage to surrounding tissues.
Innovation Solution
A method and system for reconstructing a 3D anatomical structure using a conditional variational autoencoder to predict deformations from partial 2D images, applying spatial transformations to generate motion-compensated 3D volumes, enabling real-time tracking of non-rigid motion without requiring ground truth registration fields or segmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 2D slice imaging is used for image-guided radiotherapy, then the imaging process is simple and fast, but the out-of-plane motion of internal anatomy cannot be captured, leading to inaccurate target tracking
Solution Approach 1:
The patent uses a neural network to generate a synthetic 3D volumetric image from 2D image slices. The neural network learns the mapping between 2D images and 3D anatomy, creating a virtual 3D representation that captures out-of-plane motion without requiring actual 3D imaging hardware. This copying approach maintains the simplicity of 2D imaging while achieving 3D reconstruction capabilities.
Solution Approach 2:
The patent replaces the mechanical/physical 3D imaging system with a computational neural network model. Instead of using complex 3D imaging hardware to directly capture volumetric data, the system uses 2D images fed into a trained neural network that computationally reconstructs the 3D anatomy and motion. This substitution eliminates the need for complex 3D imaging equipment while achieving accurate 3D reconstruction.
2Measurement precision
If 3D volumetric imaging is used to capture non-rigid motion, then accurate target tracking is achieved, but the imaging time and computational resources increase significantly
Solution Approach 1:
The patent trains the neural network in advance using a large dataset of paired 2D and 3D images from multiple respiratory phases. This preliminary training allows the model to learn the complex mapping between 2D images and 3D anatomy, including non-rigid motions. During actual use, the pre-trained network can quickly reconstruct 3D volumes from 2D images without requiring time-consuming real-time 3D imaging or complex computational registration processes.
Solution Approach 2:
The neural network creates a synthetic 3D volumetric representation by copying and transforming 2D image information. Rather than capturing actual 3D data through complex imaging sequences, the system generates a virtual 3D model by processing multiple 2D slices through the neural network, significantly reducing imaging time while maintaining 3D reconstruction accuracy.
3Measurement precision
If traditional image registration methods are used to track anatomical motion, then ground truth registration fields are required, but this increases the complexity and reduces the applicability to real-time free-breathing therapies
Solution Approach 1:
The patent replaces traditional image registration methods with a neural network that directly maps 2D images to 3D volumetric representations. Instead of requiring ground truth registration fields and performing complex iterative registration algorithms, the neural network learns the transformation directly from training data, eliminating the need for ground truth references and simplifying the overall system architecture.
Solution Approach 2:
The patent substitutes the mechanical/computational image registration process with a neural network-based approach. Rather than using traditional registration algorithms that require ground truth and perform complex iterative optimizations, the system uses a pre-trained neural network that directly converts 2D images into 3D representations, significantly reducing computational complexity and enabling real-time application.
Data Source
AI summary
There are described methods and systems for reconstructing a 3D anatomical structure undergoing non-rigid motion. The method comprises obtaining a 3D reference volume of the anatomical structure of the body, the reference volume corresponding to the anatomical structure at a reference phase of a respiratory cycle; acquiring 2D images of the anatomical structure at m prior times Tin={t−m, . . . , t−2, t−1}; estimating a set of deformations of the 3D reference volume at times n future Tout={t, t+1, . . . , t+n} from a previously learned probability distribution conditioned on partial observations and anatomical information; applying a spatial transformation to the 3D reference volume based on the set of deformations; and displaying the reference volume post-spatial transformation as a motion-compensated anatomical structure for each time step iϵTout.