4D Scene Reconstruction via Neural Sinogram Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computed tomography (CT) techniques struggle to accurately represent the dynamics of moving or deforming objects due to limited projections, which can result in artifacts and blurry edges, especially when the object is dynamic, such as a beating heart, and require extensive data collection over multiple rotations.
Innovation Solution
A 4D scene reconstruction system that iteratively generates a sequence of 3D representations over time, adjusting weights and motion fields to synthesize views that match collected data, using machine learning techniques like convolutional neural networks and implicit neural representations to optimize the representation of motion without a training phase.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If conventional CT techniques are used to collect limited projections, then data acquisition time is reduced, but image quality deteriorates with visible artifacts and blurry edges
Solution Approach 1:
The patent uses a neural network to learn the mapping from limited projections to high-quality 3D images by training on synthetic data where full projections are available. The network creates a copy of the complete imaging process, allowing limited projections to be transformed into high-quality reconstructions without requiring actual full-rotation data collection.
Solution Approach 2:
The system performs preliminary training in silico using synthetic data generated from simulations with known ground truth. This pre-training phase prepares the neural network to handle limited projection scenarios, so that when actual limited projections are collected, the network is already optimized to produce high-quality images without requiring extensive real-world calibration.
2Measurement precision
If multiple rotations are used to collect comprehensive views, then image accuracy is improved, but data acquisition time and complexity increase
Solution Approach 1:
The neural network learns to replicate the effect of multiple-rotation data collection by processing limited projections through trained filters and feature extractors. Instead of physically rotating the scanner multiple times, the network copies the information that would be obtained from multiple rotations using only the limited available projections.
Solution Approach 2:
The system changes the parameter space by transforming the problem from physical rotation angles to neural network feature spaces. The network learns optimal parameter transformations that map limited projection data to accurate 3D reconstructions, effectively changing how measurement precision is achieved from mechanical rotation to computational transformation.
3Measurement precision
If high current X-ray sources are used to increase flux, then measurement quality is improved, but patient dose increases
Solution Approach 1:
The neural network copies the measurement enhancement effect of high-current sources through learned feature extraction and image processing. Instead of physically increasing X-ray flux to improve measurement quality, the network processes low-flux measurements through trained layers that replicate the quality enhancement that would result from higher flux, thereby avoiding increased patient dose.
Solution Approach 2:
The patent replaces the mechanical/physical approach of increasing X-ray source current with a computational approach using neural networks. The physical system (high-current source) is substituted with an information processing system that achieves the same measurement quality improvement through learned transformations rather than increased energy delivery.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system effectively generates a 4D representation of dynamic scenes with improved accuracy and reduced artifacts, capable of representing motion without the need for extensive data collection, applicable in various fields like medical imaging and material science.
Implementation Method 1
CT employs an X-ray source and an X-ray detector. The X-ray source transmits X-rays through the object with an initial intensity, and the X-ray detector, which is on the opposite side of the object from the source, measures the final intensities of the X-rays that pass through the object
Implementation Method 2
CT is a technique that noninvasively generates cross-sectional images (or views) of the linear attenuation coefficients (LACs) of materials in an object of interest
Data Source
AI summary
A system for generating a 4D representation of a scene in motion given a sinogram collected from the scene while in motion. The system generates, based on scene parameters, an initial 3D representation of the scene indicating linear attenuation coefficients (LACs) of voxels of the scene. The system generates, based on motion parameters, a 4D motion field indicating motion of the scene. The system generates, based on the initial 3D representation and the 4D motion field, a 4D representation of the scene that is a sequence of 3D representations having LACs. The system generates a synthesized sinogram of the scene from the generated 4D representation. The system adjusts the scene parameters and the motion parameters based on differences between the collected sinogram and the synthesized sinogram. The processing is repeated until the differences satisfy a termination criterion.