Depth-Based Patient Exit Prediction With Privacy Preservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in preventing the inadvertent release of personally-identifiable information (PII) while monitoring environments, particularly in medical settings, due to the capture of RGB and IR image data that may contain sensitive information.
Innovation Solution
A neural network is trained using annotated depth data, which is processed off-site to mitigate PII, and then associated with corresponding RGB and IR image frames to predict events like a patient exiting a support surface, employing techniques like federated learning and differential privacy to safeguard data privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If RGB and IR image data are used for monitoring, then monitoring accuracy is improved, but personally-identifiable information is exposed
Solution Approach 1:
The patent extracts only the depth information from the multi-modal image data (RGB and IR) while discarding the visually identifiable features. By using depth image data as the primary input for the neural network, the system obtains precise spatial and temporal information about patient movement and exit events without capturing personally-identifiable visual characteristics.
Solution Approach 2:
The patent creates a depth-based representation copy of the patient's position and movement patterns without using the original RGB/IR image data. The depth map serves as an alternative representation that preserves monitoring functionality while eliminating privacy risks associated with visual identification.
2Object-affected harmful factors
If depth image data is used alone, then privacy is preserved, but prediction accuracy may deteriorate
Solution Approach 1:
The patent merges depth image data with annotated training data that was originally derived from multi-modal inputs. By combining the privacy-preserving depth data with richly annotated training examples (which may have used RGB/IR data during the annotation phase), the system achieves both privacy protection and high prediction accuracy.
Solution Approach 2:
The patent replaces the need for complex multi-sensor fusion mechanisms with a simplified depth-based neural network approach. Instead of mechanically combining RGB, IR, and depth data streams, the system uses a single depth input channel that has been optimized through training with annotated data, achieving comparable or superior accuracy with enhanced privacy.
3Reliability
If neural networks are trained with multi-modal data, then prediction performance is improved, but data processing complexity increases
Solution Approach 1:
The patent segments the training process into distinct phases: (1) annotation phase using multi-modal data to create ground truth labels, and (2) training phase using only depth data with the pre-created annotations. This segmentation allows the system to benefit from rich multi-modal annotations without requiring complex multi-modal processing during actual operation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system effectively predicts events while minimizing the exposure of sensitive information, ensuring privacy preservation by leveraging depth data annotation and advanced training methods.
Implementation Method 1
Some cameras capture depth frame data by transmitting a near-infrared light over a portion of the camera's field-of-view and determine a time of flight (TOF) associated with the transmitted light.
Implementation Method 2
Some cameras also capture infrared images by detecting and measuring the infrared energy of objects within the camera's field-of-view.
Data Source
AI summary
A system comprises a computer including one or more processors and memory. The memory includes instructions such that the one or more processors are programmed to receive at least one of Red-Green-Blue (RGB) image data or infrared (IR) image data from one or more cameras communicatively connected to the one or more processors, where the RGB image data and the IR image data represents an environment including at least one individual disposed along a support surface. The one or more processors receive annotated depth image data that corresponds to the RGB image data and the IR image data. The one or more processors train a neural network with at least one of the RGB image data and the IR image data corresponding to the annotated depth image data, where the neural network is trained to predict when the individual is exiting the support surface.