Operating Room Video Anonymization with Ensemble Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing face detection techniques in operating room (OR) videos are ineffective due to faces being heavily covered by personal protective equipment (PPE) and occluded by staff and equipment, leading to challenges in accurately detecting and de-identifying patient and OR personnel faces.
Innovation Solution
A multi-staged deep-learning approach combining top-down face detection with a temporal-based face tracker that utilizes contextual information and temporal correlations between frames to enhance face detection, reducing flickering bounding boxes and improving detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing bottom-up face detection techniques are used, then the system is simple to implement, but detection accuracy deteriorates due to PPE coverage and occlusion
Solution Approach 1:
The detection system is divided into multiple independent modules: a top-down face detection module for initial face location, a bottom-up face detection module for feature-based detection, and a temporal tracking module for continuity. Each module handles specific aspects of face detection, allowing them to complement each other's strengths and mitigate individual weaknesses.
Solution Approach 2:
The patent merges multiple detection approaches (top-down and bottom-up) into a unified system that combines their results. The top-down module provides robustness against occlusion while the bottom-up module handles visible features, and their combined output is processed by the temporal tracking module to achieve high accuracy despite PPE coverage.
2Reliability
If top-down face detection is used, then detection robustness improves against occlusion, but false negatives increase in low-resolution or off-angle faces
Solution Approach 1:
The system segments the detection task into two independent paths: top-down detection for robustness and bottom-up detection for precision. Each path processes faces differently according to their characteristics, with the top-down path handling occluded faces and the bottom-up path handling visible feature-based detection.
Solution Approach 2:
Different detection strategies are applied to different regions and face types. The top-down module focuses on contextual information and body pose for robust detection, while the bottom-up module focuses on facial features for precise detection. The temporal tracking module then refines results based on local temporal patterns.
3Reliability
If single-frame face detection is used, then processing speed is fast, but detection reliability deteriorates due to flickering bounding boxes
Solution Approach 1:
The system performs preliminary face detection on multiple frames before final determination. By detecting faces in advance across temporal sequences and using tracking to maintain consistency, the system can confidently identify faces even when single-frame detection is uncertain, reducing false negatives without significant time penalty.
Solution Approach 2:
The temporal tracking module maintains continuous monitoring of detected faces across video frames. Instead of independent single-frame analysis, the system continuously tracks face positions and characteristics over time, ensuring that detection reliability is maintained through temporal consistency while processing proceeds efficiently.
Data Source
AI summary
Disclosed are various face-detection and human de-identification systems and techniques based on deep learning. In one aspect, a process for de-identifying people captured in an operating room (OR) video is disclosed. This process can begin by receiving a sequence of video frames from an OR video. Next, the process applies a first machine-learning face detector based on a first deep-learning model to each video frame in the sequence of video frames to generate a first set of detected faces. The process further applies a second machine-learning face detector to the sequence of video frames to generate a second set of detected faces, wherein the second machine-learning face detector is constructed based on a second deep-learning model different from the first deep-learning model. The process subsequently de-identifies the received sequence of video frames by blurring out both the first set of detected faces and the second set of detected faces.