Surgical Video Phase Recognition with Self-Knowledge Distillation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classifying content in surgical videos is challenging due to the long duration of videos, ambiguous boundary frames, and data imbalance, leading to difficulties in capturing temporal dependencies and accurate phase recognition.
Innovation Solution
Implement a self-knowledge distillation framework that integrates self-knowledge distillation into both the encoder and decoder of neural networks, using joint optimization of classification and feature similarity losses to enhance feature representation and reduce over-segmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional classification models are used for surgical video content, then the model structure is simple, but the classification accuracy is low due to ambiguous boundary frames and data imbalance
Solution Approach 1:
The patent introduces a boundary frame identification module as an intermediary component between the video input and phase classification. This module specifically detects and processes ambiguous boundary frames, allowing the main classification model to focus on clear-phase segments while boundary frames are handled through separate temporal relationship analysis, thereby improving overall classification accuracy without requiring complete model restructuring
Solution Approach 2:
The patent segments the surgical video processing into distinct components: boundary frame identification, clear-phase classification, and temporal relationship analysis. This segmentation allows each component to be optimized independently, with the boundary frame module handling ambiguous cases and the classification model handling clear phases, improving accuracy while maintaining manageable model complexity
2Reliability
If the model processes entire long-duration surgical videos, then complete temporal context is captured, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent extracts and separately processes boundary frames from the complete surgical video. By identifying and isolating these critical temporal transition points, the system can focus computational resources on analyzing temporal relationships at key moments rather than processing every frame uniformly, thereby maintaining temporal context reliability while reducing overall processing time
Solution Approach 2:
The patent performs preliminary identification of boundary frames before main phase classification. This preliminary action allows the system to pre-mark temporal transition points and prepare a structured representation of temporal relationships, enabling faster subsequent classification processing without losing temporal dependency information
3Measurement precision
If more training data is used to handle data imbalance, then classification accuracy improves, but training time and computational resources increase
Solution Approach 1:
The patent applies local quality enhancement by specifically targeting boundary frame samples for improved processing and representation. Instead of uniformly processing all training data, the system identifies boundary frames as having special importance and applies focused processing strategies to these local regions of the data, improving overall classification accuracy while avoiding the computational cost of processing every sample with equal intensity
Data Source
AI summary
Examples described herein provide a computer-implemented method that includes performing training of a self-knowledge distillation encoder and a self-knowledge distillation decoder for video frames of a surgical procedure. A trained version of the self-knowledge distillation encoder and the self-knowledge distillation decoder can be combined as a phase recognition model to predict surgical phases of the surgical procedure in one or more videos.