3D Instrument-to-Object Localization From 2D X-Ray Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surgical procedures for determining 3D relative positions and orientations between instruments and target objects in orthopedic or spinal surgery are challenging, especially when the target object's geometry is unknown or the instrument is not uniquely localizable in 2D X-ray images, leading to time-consuming and inaccurate procedures like distal locking of long antegrade intramedullary nails.
Innovation Solution
A system utilizing artificial intelligence, such as deep morphing and neural networks, processes intraoperative 2D X-ray images to determine 3D representations and relative orientations without additional hardware, by combining knowledge of X-ray image generation processes and utilizing a priori geometric information to estimate imaging depth and align instruments accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional iterative C-arm positioning is used to achieve true lateral view for distal locking, then the hole appears round in X-ray image, but the process is time-consuming requiring 5 to 20 X-ray images and re-adjustments
Solution Approach 1:
The system performs preliminary 3D reconstruction of the target object (locking hole) and instrument (drill) positions and orientations before the actual distal locking procedure. By pre-determining the optimal C-arm positioning angles based on 3D models and a priori geometric information, the system eliminates the need for iterative trial-and-error positioning, reducing the number of X-ray images required from 5-20 to just 1-2 images while maintaining high positioning accuracy.
2Loss of time
If fluoroscopic mode is used to achieve faster positioning, then positioning time is reduced, but X-ray dose increases
Solution Approach 1:
The system calculates the optimal C-arm positioning angles in advance using 3D reconstruction and a priori geometric information about the target object and instrument. This preliminary determination allows the surgeon to acquire the correct X-ray image in a single or few static images rather than using continuous fluoroscopic imaging, thereby reducing positioning time while simultaneously minimizing the cumulative X-ray dose to the patient.
3Ease of operation
If drilling is performed without precise 3D localization, then the procedure is simpler, but accuracy of drilling direction deteriorates
Solution Approach 1:
The system replaces complex mechanical tracking systems and reference bodies with an AI-based computer vision approach. By using deep learning algorithms to automatically detect and classify anatomical landmarks and instrument positions in 2D X-ray images, and by performing 3D reconstruction based on a priori geometric information, the system achieves precise 3D localization without requiring additional hardware, thereby maintaining surgical simplicity while significantly improving drilling accuracy.
4Measurement precision
If reference bodies or additional hardware are used for 3D localization, then localization accuracy improves, but device complexity increases
Solution Approach 1:
The system extracts and utilizes a priori geometric information inherently present in the surgical setup, such as the known geometry of the intramedullary nail, locking hole dimensions, and drill specifications. By leveraging this pre-existing geometric knowledge combined with AI-based 2D image analysis, the system achieves accurate 3D localization without extracting or adding physical reference bodies or tracking hardware, thereby maintaining localization accuracy while eliminating additional hardware complexity.
Data Source
AI summary
Systems and methods are provided for processing X-ray images, wherein the methods are implemented as a software program product executable on a processing unit of the systems. Generally, an X-ray image is received by the system, the X-ray image being a projection image of a first object and a second object. The first and second objects are classified, and a respective 3D model of the objects is received. At the first object, a geometrical aspect like an axis or a line is determined, and at the second object, another geometrical aspect like a point is determined. Finally, a spatial relation between the first object and the second object is determined based on a 3D model of the first object, a 3D model of the second object, and the information that the point of the second object is located on the geometrical aspect of the first object.