Surgical State Detection via Simulated Image Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer-assisted surgical systems face challenges in processing dynamic and unpredictable surgical environments, making it difficult to collect diverse training data for machine-learning models to accurately detect and interpret unpredictable events during procedures.
Innovation Solution
The system generates virtual images using simulated data with varying perspectives, camera poses, lighting, and object motion, and trains machine-learning models using these images to identify surgical tools, anatomical objects, and actions, enabling real-time processing of live data streams for enhanced procedural state detection and action guidance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real surgical data is collected for training machine-learning models, then model accuracy may improve, but data diversity and coverage of unpredictable events remain insufficient
Solution Approach 1:
The patent creates virtual copies of surgical environments through simulated images that replicate real surgical scenes, tools, and anatomical structures. These synthetic training images are generated by rendering engines that copy the visual characteristics and spatial relationships of actual surgical procedures, providing diverse training data without requiring extensive real-world data collection
Solution Approach 2:
The patent systematically varies multiple parameters in the virtual surgical environment including camera angles, lighting conditions, tool positions, anatomical variations, and procedural states. By changing these parameters across numerous simulated scenarios, the training data achieves diversity comparable to or exceeding what could be collected from real surgeries alone
2Loss of time
If machine-learning models are trained on limited real data, then training time is reduced, but detection reliability and confidence in unpredictable events deteriorate
Solution Approach 1:
The patent performs preliminary training actions by pre-generating extensive virtual training datasets covering rare and unpredictable surgical events before deploying the model. This advance preparation ensures the model encounters diverse scenarios during training, improving reliability without extending real-time training duration
Solution Approach 2:
The patent uses synthetic copying of surgical events to create comprehensive training scenarios for rare and unpredictable situations. By replicating various surgical outcomes, complications, and unexpected events in virtual environments, the model learns to recognize and respond to these scenarios with high confidence
3Adaptability or versatility
If diverse training data is collected from multiple surgical sources, then model adaptability improves, but data processing complexity and system requirements increase
Solution Approach 1:
The patent replaces complex multi-source data collection and processing systems with a unified virtual rendering system. Instead of ingesting and processing diverse real-world datasets from multiple surgical sources, the system generates all necessary training data through controlled synthesis, dramatically simplifying the data pipeline while maintaining or improving adaptability
Data Source
AI summary
A set of virtual images can be generated based on one or more real images and target rendering specifications, such that the set of virtual images correspond to (for example) different rendering specifications (or combinations thereof) than do the real images. An image style can be transferred to the at least some of the virtual images of the set of virtual images to generate a stylized virtual image. A machine-learning model can be trained using a plurality of stylized virtual images. Another real image can then be processed using the trained machine-learning model. The processing can include segmenting the other real image to detect whether and/or which objects are represented (and/or a state of the object). The object data can then be used to identify (for example) a state of a procedure.