Depth Camera Registration for Intraoperative 3D Bone Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for obtaining intraoperative 3D geometric representations of patient anatomy during surgery are cumbersome, requiring specialized scanners or lengthy manual processes, and lack efficient registration techniques.
Innovation Solution
A method using a depth camera and registration device with a tracker, combined with a localization system, to acquire and register 3D point clouds from multiple positions, filtering and transforming them into a unified patient coordinate system for accurate bone tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If specialized scanners (CT, MRI, CBCT, ultrasound) are used to obtain intraoperative 3D geometric representation, then measurement precision is improved, but device complexity and loss of time increase
Solution Approach 1:
The patent uses a depth camera to capture optical images of the patient anatomy and creates a 3D geometric representation (point cloud) as a copy of the actual anatomy. This optical copy replaces the need for specialized medical scanners, achieving comparable measurement precision without the complexity of CT, MRI, or ultrasound equipment.
Solution Approach 2:
The patent replaces mechanical/scanned-based imaging systems (CT, MRI, ultrasound scanners) with an optical system (depth camera). The depth camera uses optical fields and image processing to generate 3D geometric data, substituting complex mechanical scanning mechanisms with a simpler optical capture and computational approach.
2Device complexity
If manual palpation or hand-held camera is used for intraoperative data acquisition, then device complexity is reduced, but loss of time and productivity decrease
Solution Approach 1:
The patent employs a robotic arm that can dynamically position and reposition the depth camera around the patient anatomy automatically. This dynamic positioning system replaces manual palpation and hand-held camera operations, enabling rapid data acquisition from multiple angles without manual intervention, thereby significantly improving productivity while keeping the core imaging device simple.
Solution Approach 2:
The system performs automated 3D reconstruction and registration processes without requiring manual operation. The robotic arm automatically moves the camera, the system automatically captures images from multiple positions, and the processing unit automatically reconstructs the 3D geometry and registers it with preoperative data, making the entire workflow self-service and highly efficient.
3Measurement precision
If multiple positions around the region of interest are acquired, then measurement precision and completeness improve, but loss of time increases
Solution Approach 1:
The system performs preliminary registration by capturing an image of the registration device at each position and using the registration device tracker to establish coordinate transformations before final 3D reconstruction. This preliminary action at each position enables efficient data integration and reduces the time needed for post-processing, allowing multiple positions to be acquired without proportionally increasing total time.
Solution Approach 2:
The robotic arm continuously moves the depth camera around the patient anatomy, capturing images at multiple positions in a continuous sequence without interruption. The system maintains continuous data acquisition and processing, transforming the 3D coordinates and registering data in real-time, thereby achieving complete 3D representation from multiple positions without the time loss associated with stopping and starting acquisitions.
4Manufacturing precision
If registration device with characteristics elements is used, then manufacturing precision of registration is improved, but device complexity increases
Solution Approach 1:
The registration device includes characteristics elements with distinct visual features (such as colored markers or reflective elements) that can be easily detected and recognized by the depth camera. These visual characteristics enable precise identification and tracking of the registration device position and orientation, improving registration accuracy without requiring complex mechanical structures.
Solution Approach 2:
The characteristics elements on the registration device are designed with asymmetric geometries or distinct shapes that facilitate unique identification and orientation determination. This asymmetric design allows the system to accurately determine the registration device's pose (position and orientation) in 3D space, improving manufacturing precision of registration while keeping the device structure relatively simple.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The invention concerns a computer implemented method, comprising the steps of: - positioning, at several positions around a region of interest (ROI) of a patient (1), an assembly (3) comprising a depth camera (4) rigidly attached to a registration device (5), the registration device (5) being disposed in the field of view of the depth camera (4), the registration device (5) comprising a registration device tracker (9); - acquiring, by means of the depth camera (4), for each position of a plurality of positions around the region of interest, a set of points comprising a first cloud of 3D points representing the region of interest and a second cloud of 3D points representing the registration device; - locating, for each position, the registration device tracker (9) relatively to a patient tracker (8) attached to the patient (1) by means of a locating system (7); - obtaining, for each position, from the location of the registration device tracker (9) relatively to the patient tracker of a transformation ™TPM from a coordinate system of the registration device tracker (9) to a coordinate system of the patient tracker; - determining a transformation CTMP from a coordinate system of the depth camera to a coordinate system of the patient, the transformation being computed by the following equation CTPM= CTTP TPTTM ™TPM with C a coordinate system attached to the depth camera, PM a coordinate system attached to the patient tracker, TM a coordinate system attached to the registration device tracker (9), TP a coordinate system attached to the registration device (5); - processing, from the transformation CTMP at each position, each set of points in order to obtain, in the coordinate system attached to the patient tracker, a corresponding cloud of 3D points of the region of interest (ROI); - combining each cloud of 3D points of interest in the coordinate system attached to the patient tracker for obtaining a resulting cloud of 3D points of the region of interest representing the region of interest.