Surgical Robotic Exercise Digitization With Selective 3D Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surgical robotic systems lack a standardized method for digitizing surgical procedures to facilitate uniformity, pre-operative planning, post-operative analysis, and intra-operative guidance, leading to challenges in data storage and review of terabytes of raw sensor data.
Innovation Solution
Integration of depth cameras and untethered electronic devices with surgical robotic systems to create a common coordinate frame, using semantic segmentation and machine learning for selective data storage, and generating a digital asset that reconstructs surgical exercises for playback and real-time alerts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If raw sensor data from surgical robotic systems is stored for complete surgical exercise replay, then measurement precision and data completeness are improved, but data storage volume increases to terabytes making storage and review difficult
Solution Approach 1:
The patent extracts only the essential information from raw sensor data by identifying and storing only significant surgical events and keyframe images rather than continuously storing all raw data. This extraction process reduces storage volume to manageable levels while preserving the critical information needed for surgical exercise replay and analysis.
Solution Approach 2:
The patent segments the continuous surgical procedure into discrete significant events and keyframes. By dividing the continuous data stream into meaningful segments (surgical events, key moments), the system stores only these segmented portions rather than the entire continuous raw data, reducing storage requirements while maintaining replay accuracy.
2Measurement precision
If extensive data collection is undertaken to find commonalities between surgeons from different institutions, then measurement precision for surgical standardization is improved, but device complexity and data processing requirements increase
Solution Approach 1:
The patent implements automated data processing and analysis where the system itself performs the complex tasks of data collection, event identification, and comparison across different surgical exercises. The automated event detection and keyframe selection algorithms handle the complexity without requiring manual intervention, enabling precise surgical standardization analysis while managing system complexity through automation.
Solution Approach 2:
The patent changes the parameters of data representation by transforming raw sensor data into standardized event representations and keyframe images. This parameter transformation simplifies the data structure and enables easier comparison and analysis across different surgical exercises from various institutions, reducing the complexity of processing while maintaining measurement precision.
3Measurement precision
If depth cameras and multiple sensors are integrated with surgical robotic systems to create comprehensive digital records, then measurement precision for surgical digitization is improved, but device complexity increases
Solution Approach 1:
The patent makes the sensor integration system universal by designing a unified architecture where depth cameras, RGB cameras, and robotic sensors all serve the common function of creating a digital record of surgical exercises. The system uses a common coordinate frame and standardized processing pipeline that handles multiple sensor types, improving digitization accuracy while managing complexity through multi-functionality.
Solution Approach 2:
The patent introduces an intermediary processing layer that mediates between the multiple sensors and the final digital record. This intermediary layer (including the common coordinate frame system and event detection algorithms) harmonizes data from different sensor sources, improving measurement precision while shielding the user from the underlying complexity of sensor integration.
4Measurement precision
If continuous RGBD point cloud data is fused with robot data to create common coordinate frame, then measurement precision for spatial reconstruction is improved, but data storage volume and processing requirements increase
Solution Approach 1:
The patent extracts only the essential spatial information needed for reconstruction by identifying keyframes and significant events rather than storing every continuous point cloud frame. This extraction maintains spatial reconstruction accuracy for the critical moments while dramatically reducing the volume of data that needs to be stored and processed.
Solution Approach 2:
The patent segments the continuous spatial data into discrete keyframe images and event markers. By dividing the continuous point cloud stream into meaningful segments (key moments in the surgical procedure), the system preserves spatial reconstruction accuracy for important events while reducing overall data storage requirements.
Data Source
AI summary
A surgical exercise performed with a surgical robotic system is sensed by depth cameras, generating 3D point cloud data. Robot system data associated with the surgical robotic system is logged. Object recognition is performed on image data produced by the one or more depth cameras, to recognized objects, including surgical equipment and people, in the operating room (OR). The surgical exercise is digitized by storing the 3D point cloud data of unrecognized objects, a position and orientation associated with the recognized objects, and c) the robot system data.