3D Joint Modeling From Monocular Arthroscopy for Bone Tunnel Guidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surgical technologies lack effective real-time guidance for making bone tunnels during arthroscopic surgeries, particularly in ligament reconstruction, due to limited visibility and the difficulty in accurately positioning surgical tools within the joint.
Innovation Solution
A computer-implemented method using monocular intraoperative two-dimensional images and a trained learning model to calculate a partial three-dimensional model of the joint, combining depth maps and imaging device position data to guide surgeons in drilling bone tunnels, without relying on Lidar, RGB-D cameras, or stereoscopic systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If monocular two-dimensional images are used for real-time joint modeling, then device complexity is reduced and ease of operation is improved, but measurement precision and manufacturing precision of the three-dimensional model deteriorate
Solution Approach 1:
The patent introduces depth maps as an intermediary representation that bridges the gap between monocular 2D images and 3D reconstruction. The learning model generates depth maps from single 2D images, providing pseudo-depth information that enables accurate 3D joint modeling without requiring complex multi-camera or active sensing systems. This intermediary depth map approach resolves the contradiction by enabling precise measurement through a computationally generated intermediate representation rather than direct physical measurement.
Solution Approach 2:
The patent replaces traditional mechanical/optical depth sensing systems (such as stereo cameras, LiDAR, or structured light systems) with a learning-based computational approach. Instead of using multiple physical sensors or active illumination to obtain depth information, the system uses a trained neural network to infer depth from monocular images. This substitution of mechanical sensing with intelligent algorithms achieves comparable or superior precision while dramatically reducing device complexity.
2Ease of operation
If monocular images are used instead of stereoscopic or active sensing systems, then ease of operation is improved, but reliability of depth prediction deteriorates
Solution Approach 1:
The patent applies preliminary action by training the learning model extensively before actual surgical use. The model is trained on large datasets of labeled joint images with ground truth depth information, enabling it to make reliable predictions during surgery without requiring complex real-time sensing. This pre-computation and pre-training phase ensures that when the system is deployed, it can reliably predict depth from monocular images despite the simplicity of the input data.
Solution Approach 2:
The system incorporates feedback mechanisms where the learning model continuously refines depth predictions based on the evolving 3D joint model and observed image sequences. The SLAM component provides feedback by tracking camera motion and updating the joint model consistency, which in turn improves depth prediction reliability. This closed-loop feedback ensures that even though monocular images lack direct depth information, the system maintains high reliability through iterative refinement and consistency checking.
3Productivity
If real-time three-dimensional modeling is implemented during surgery, then productivity is improved through better guidance, but loss of time increases due to processing requirements
Solution Approach 1:
The patent implements periodic action by updating the three-dimensional joint model at discrete time intervals rather than continuously processing every single image frame. The system processes images at optimized intervals, generating depth maps and updating the SLAM model periodically, which reduces computational overhead while maintaining real-time guidance capability. This periodic updating strategy balances productivity improvement with minimizing processing time loss.
Solution Approach 2:
The system applies partial action by focusing computational resources on modeling only the relevant portions of the joint anatomy that are currently in view or about to be operated on. Rather than reconstructing the entire joint model at full resolution continuously, the system dynamically adjusts the modeling scope and detail level based on surgical progress and camera position, reducing unnecessary processing while maintaining guidance accuracy where needed.
Data Source
AI summary
The invention relates to a computer-implemented method and an associated device for modelling a joint of a patient for real-time assistance in performing at least one bone tunnel by arthroscopy in said joint, said method comprising: receiving a stream of intraoperative monocular two-dimensional images obtained using an imaging device, each of the intraoperative monocular two-dimensional images being obtained at a time t and comprising at least a region of interest of the joint; for each intraoperative monocular two-dimensional image, calculating a depth map associated with time t by implementing a first previously trained learning model, said first learning model being configured to receive as input at least said intraoperative monocular two-dimensional image obtained at time t; calculating a current partial three-dimensional model of the patient's joint associated with a current time tc, on the basis of at least one depth map associated with the current time tc, at least one depth map associated with a previous time tc−Δt relative to the current time tc, localization information of the imaging device, and a preoperative three-dimensional model of the patient's joint.