Synthetic 2D Image Generation for Automated 3D Registration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accurate registration of preoperative 3D image data with real-time 2D fluoroscopy data during surgery is challenging, especially when identifying anatomical features like vertebrae is difficult, leading to increased procedural time and potential errors.
Innovation Solution
The method involves pre-processing 3D data to generate synthetic 2D images from various viewing directions, extracting feature data using techniques like the generalized Hough transform, and matching these features with real-time 2D images to automatically determine the registration start position, thereby aligning 2D and 3D data accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual vertebra identification is used for registration, then the registration process can be performed, but the time taken to achieve registration increases and accuracy decreases when vertebrae are not easily visible
Solution Approach 1:
The system performs preliminary processing of the 3D image data to generate synthetic 2D images and extract feature information before the actual registration process. This pre-computation of feature data from multiple viewing angles allows the system to quickly match features during registration without manual intervention, thereby reducing registration time while maintaining accuracy.
Solution Approach 2:
The system replaces the manual mechanical process of visual inspection and identification with an automated computer-based image processing and pattern recognition system. The automated feature extraction and matching algorithms substitute for the manual mechanical action of a technician visually identifying vertebrae, eliminating time loss and improving consistency.
2Reliability
If manual visual inspection is used to identify vertebrae, then registration can be attempted, but reliability decreases when anatomical features are not easily visible
Solution Approach 1:
The system generates synthetic 2D images from the 3D data at multiple viewing angles and dimensions. By examining features across multiple dimensional perspectives rather than a single 2D view, the system can reliably detect and identify vertebrae even when they are not easily visible in the original fluoroscopy image, thereby improving registration reliability.
Solution Approach 2:
The system creates synthetic copies of the 3D image data projected onto 2D planes from various angles. These synthetic 2D images serve as reference copies that can be compared with the actual fluoroscopy image, enabling reliable feature detection even when the original image quality is poor or features are not easily visible.
3Productivity
If automated feature extraction is implemented, then registration time is reduced and accuracy is improved, but system complexity increases
Solution Approach 1:
The system divides the complex registration task into separate modular components: synthetic image generation, feature extraction, feature matching, and registration. Each module performs a specific function independently, which manages system complexity by breaking down the overall process into manageable segments while maintaining high productivity through automated processing.
Data Source
Figure 1
Figure 2
Figure 3(a)~3(c)
AI summary
Embodiments of the invention provide a system and method that is able to automatically provide a starting point for 2D to 3D image registration, without relying on human recognition of features shown in the 2D image. This is achieved by pre-processing the 3D data to obtain synthetically generated 2D images of those parts of the 3D data volume which will be used for registration purposes. Many different synthetically generated 2D images of the or each part of the 3D volume are produced, each from a different possible viewing direction. Each of these synthetic images is then subject to a feature extraction process to extract characterising feature data of the registration feature shown in the images. Once the feature extraction has been undertaken for each image, when registration is to be performed the real-time 2D image is processed by applying each of the sets of extracted features thereto, to try and identify which set best matches the registration features in the 2D image. For example, where a generalised Hough transform was used in the feature extraction, the R tables would be applied to the 2D image to obtain respective accumulation images. The accumulation images may then be ranked to identify which registration feature is shown in the 2-D image, and from which view direction. This gives the required information of which registration feature is being shown in the 2D image, and also the in-plane location and orientation. This information can then be used as a starting point for the 2D to 3D registration procedure.