Streaming Vision Architecture With Early-Discard Privacy Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing wearable cameras for human studies face challenges in achieving compactness, long system lifetime, high performance, and privacy protection due to large form factors and short battery life, particularly when capturing sensitive information.
Innovation Solution
The NIR-sighted architecture employs a non-visual imager to generate obfuscation masks on the fly, enabling frame-level and pixel-level early-discard of irrelevant video data, using a low-power FPGA compressor to reduce memory and computational requirements, allowing for compact, privacy-enhancing, and long-lasting wearable cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the camera captures and stores all video data at high resolution, then the system performance and image quality are improved, but the memory requirements and device complexity increase, leading to larger form factors
Solution Approach 1:
The patent extracts only the relevant portions of video data based on depth information from the auxiliary sensor. The main processor identifies regions of interest and extracts only those pixels that need to be stored at full resolution, while discarding irrelevant pixels. This extraction principle resolves the contradiction by maintaining high resolution for necessary areas while reducing overall data volume and device size.
Solution Approach 2:
The patent applies different quality levels to different regions of the video stream. Regions identified as important (e.g., containing people or objects of interest) are stored at full high resolution, while other regions are discarded or stored at lower quality. This local quality differentiation allows the system to maintain high measurement precision where needed while reducing overall device complexity and form factor.
2Loss of information
If the camera stores all video data without discrimination, then no information is lost, but the battery consumption increases and system lifetime decreases
Solution Approach 1:
The patent performs preliminary analysis of the video stream using depth information from an auxiliary sensor before final storage decisions are made. The system pre-identifies regions of interest and pre-determines which pixels will be stored, allowing for efficient memory management and reduced power consumption during the actual storage process. This preliminary action enables the system to extend battery life while maintaining data completeness for important regions.
Solution Approach 2:
The patent dynamically changes storage parameters based on scene content analysis. The system adjusts which pixels are stored, at what resolution, and when, based on real-time depth information and identified regions of interest. This parameter changing approach allows the system to minimize data storage and power consumption while ensuring that important information is preserved, thereby extending system lifetime.
3Productivity
If the camera uses a powerful processor to handle high-resolution video processing, then the system performance is improved, but the power consumption increases and device compactness is compromised
Solution Approach 1:
The patent segments the video processing task into two parts: a simple depth estimation task performed by a low-power auxiliary sensor, and a selective storage decision made by the main processor based on that depth information. This segmentation allows the majority of processing to be done by simple, low-power components while the main processor only performs lightweight decision-making about which pixels to store, thereby maintaining high productivity with low power consumption.
Solution Approach 2:
The patent introduces depth information from an auxiliary sensor as an intermediary that guides the main processor's decisions about which pixels to store. This intermediary provides the main processor with pre-filtered information about scene depth and structure, allowing it to make storage decisions without having to analyze every pixel in detail. This intermediary approach enables efficient video processing with minimal power consumption.
4Loss of energy
If the camera masks and discards video data early in the pipeline, then power consumption and memory usage are reduced, but the ability to recover or reprocess the data is lost
Solution Approach 1:
The patent implements dynamic masking where the regions to be discarded are determined in real-time based on depth information and identified regions of interest. The mask is not fixed but adapts to the current scene content, allowing the system to discard only truly irrelevant data while preserving adaptability. This dynamic approach enables the system to reduce power consumption and memory usage while maintaining the ability to reprocess or analyze the data if needs change.
Data Source
AI summary
Disclosed is a programmable streaming architecture designed for low-energy, human-centric vision applications (e.g., wearable lifelogging cameras). The disclosed device address the privacy concerns, battery life, and device size issues in existing devices. The disclosed device provides a low-power architecture for wearable cameras that allows for programmable early-discard of video frames at both frame and pixel levels. Obfuscation masks are generated on-the-fly from non-visual sensor data, enabling the device to process and store only relevant portions of video streams while discarding unnecessary data, thus enhancing privacy and extending battery life.