Multimodal Sensor Network for Privacy-Preserving Subject Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human sensing technologies face challenges in accurately detecting, counting, localizing, and identifying individuals in environments due to issues like the Correspondence Problem, which leads to ambiguous track hypotheses and privacy concerns, especially in public spaces where uninstrumented solutions are invasive and instrumented solutions require complex infrastructure.
Innovation Solution
A multimodal sensor network combining cameras and inertial sensors in wearable devices to anonymously detect and track people by fusing camera data with phone sensor information, using a two-layer system for detection, counting, and localization (DCL layer) and tracking and identification (TI layer), which leverages existing infrastructure and ensures privacy through low-power, motion-sensitive cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If uninstrumented sensing solutions are used in public spaces, then privacy is compromised through invasive monitoring, but implementation cost and infrastructure complexity are reduced
Solution Approach 1:
The system segments sensing functions into two layers: a DCL layer using uninstrumented cameras for anonymous detection, counting, and localization, and a TI layer using instrumented phones for tracking and identification. This segmentation allows privacy-preserving anonymous sensing to be separated from identification functions, enabling public space monitoring without compromising individual privacy.
Solution Approach 2:
The system introduces an intermediary approach where instrumented mobile phones act as mediators between the uninstrumented camera network and the tracking/identification system. The phones provide voluntary instrumented data that complements anonymous camera data, enabling accurate tracking while maintaining privacy since phones only track individuals who carry them and consent to the system.
2Measurement precision
If instrumented sensing solutions with wearable devices are deployed, then tracking and identification accuracy is improved, but infrastructure complexity and installation cost increase
Solution Approach 1:
The system employs self-service by leveraging existing mobile phones that users already carry, which contain inertial sensors. Instead of requiring a complex centralized infrastructure, the system utilizes the phones' built-in accelerometers and magnetometers to provide tracking data, significantly reducing infrastructure complexity while maintaining high tracking accuracy.
Solution Approach 2:
The mobile phones serve multiple functions: they act as inertial measurement units for tracking, provide wireless communication nodes, and serve as identification carriers. This multi-functionality reduces the need for dedicated tracking infrastructure, as the phones themselves become the sensing and communication infrastructure.
3Device complexity
If a unified sensing system is used, then system simplicity is maintained, but the ability to address diverse sensing needs and constraints is reduced
Solution Approach 1:
The system is dynamic and adaptive, allowing the TI layer to be activated or deactivated based on privacy requirements and application needs. The system can operate in pure anonymous mode (DCL only), hybrid mode (DCL + TI), or instrumented mode (TI only), providing flexibility to adapt to different sensing needs while maintaining a relatively simple unified architecture.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution effectively detects and tracks individuals with high accuracy, maintaining privacy while reducing installation costs and infrastructure complexity, achieving over 90% identification accuracy and localization precision in various scenarios.
Implementation Method 1
inertial sensors (such as accelerometers and magnetometers) present in wearable devices
Implementation Method 2
inertial sensors (such as accelerometers and magnetometers) present in wearable devices
Implementation Method 3
combine cameras scattered in an environment with inertial sensors
Data Source
AI summary
A multimodal sensor network is designed to extract a plurality of fundamental properties associated with subject sensing. In one aspect, such network can combine cameras distributed in an environment with inertial sensors available in subjects' wearable devices. The network can permit anonymous detection, counting, and localization of one or more subjects utilizing the cameras. In one aspect, by fusing such information with positional data from the inertial sensors contained or coupled to wearable devices associated with the one or more subjects, the network can track and can identify each subject carrying a wearable device functionally coupled to inertial sensor(s). In one aspect, the problem of subject sensing can be divided into two parts: (1) a detection, counting, and localization (DCL) layer and (2) a tracking and identification (TI) layer, wherein such layers can be implemented via simulations and a real sensor network deployment.