Cross-Enhancement Causal Transformer for Surgical Phase Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI approaches for surgical phase recognition in surgical videos rely on spatial and temporal information but struggle with accuracy and precision, particularly in online recognition, leading to suboptimal surgical outcomes due to inadequate integration of global and local information.
Innovation Solution
The implementation of a Cross-Enhancement Causal Transformer (C-ECT) with Cross-Attention Feature Fusion (CAFF) for online surgical phase recognition, which modifies transformer architectures to better integrate global and local information, using a feature extraction network like EfficientNetV2 and an action segmentation network based on transformer models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If current AI approaches utilize both spatial and temporal information for surgical phase recognition, then context awareness is improved, but accuracy and precision deteriorate due to inadequate integration of global and local information
Solution Approach 1:
The video is segmented into multiple clips with overlapping frames, where each clip is processed independently by the transformer model. This segmentation allows the system to maintain global context awareness across the entire surgical video while processing local segments with high precision, resolving the contradiction between comprehensive context utilization and accurate phase recognition
Solution Approach 2:
The patent introduces a temporal dimension to the transformer architecture by processing sequences of video clips rather than individual frames. This dimensional extension enables the model to capture both spatial relationships within frames and temporal relationships across frames, improving both context awareness and recognition accuracy simultaneously
2Loss of information
If transformer architecture is used for online surgical phase recognition, then integration of global and local information is improved, but computational complexity increases
Solution Approach 1:
By dividing the video into overlapping clips and processing them separately, the computational burden on the transformer model is reduced. Each clip requires less computational resources than processing the entire video at once, while the overlapping mechanism ensures global information is still captured through contextual propagation between adjacent clips
Solution Approach 2:
The system performs preliminary processing by extracting key features from video frames before feeding them to the transformer model. This pre-processing step reduces the dimensionality and complexity of the input data, allowing the transformer to focus on integrating global and local information without being overwhelmed by raw pixel data
Data Source
AI summary
A Cross-Enhancement Causal Transformer or simply a Cross-Enhancement Transformer (C-ECT) is described as a modification of previous transformer architectures that is suitable for online surgical phase recognition. Additionally, a Cross-Attention Feature Fusion (CAFF) is described that better integrates the global and location information in the C-ECT. This can achieve better performance on the Cholec80 dataset than the current state-of-the-art methods in accuracy and precision, recall, and in the Jaccard score. Other aspects are also described and claimed.