Cross-Modal Signal Association for Multi-Occupant Vibration Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing indoor occupant sensing systems face accuracy issues due to the challenge of establishing accurate cross-modal associations between structural vibration and wearable sensors, especially in scenarios with multiple occupants, leading to mischaracterization of activities and inefficient signal segment associations.
Innovation Solution
The implementation of a Cross-Modal Association (CMA) scheme using an Association Discovery Temporal Convolutional Network (AD-TCN) framework, which aligns and segments multimodal signals, determines association probabilities between structural vibration and wearable sensors, and estimates the association relationship between them, enhancing the accuracy of signal segment associations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple sensing modalities are used for indoor occupancy sensing, then sensing coverage and application benefits are improved, but accuracy issues arise and activities can be mischaracterized due to inaccurate cross-modal associations
Solution Approach 1:
The patent segments the sensing data into distinct temporal segments and applies separate association analysis to each segment. The AD-TCN model processes vibration and wearable sensor segments independently, identifying cross-modal associations at the segment level rather than treating the entire dataset as a single unit, thereby improving activity characterization accuracy
Solution Approach 2:
The patent introduces an intermediary association discovery mechanism that mediates between structural vibration sensors and wearable sensors. The AD-TCN model acts as an intermediary that learns the mapping relationships between these two modalities, enabling accurate cross-modal association without direct interference between the sensing devices
2Measurement precision
If cross-modal association schemes are implemented between structural vibration and wearable sensors, then association accuracy is improved, but computational complexity and model training requirements increase
Solution Approach 1:
The patent replaces complex manual association methods with a deep learning-based AD-TCN model that automatically learns cross-modal associations. The model substitutes traditional signal processing and manual feature matching with neural network-based pattern recognition, achieving higher accuracy while managing complexity through automated learning
Solution Approach 2:
The patent changes the parameter representation by using temporal convolutional networks that process sensor data in transformed temporal domains. The model learns optimal parameter representations during training, transforming raw sensor signals into features that are more suitable for association analysis, thereby improving accuracy while managing computational complexity
3Measurement precision
If signal segments from two modalities are associated in co-living scenarios, then occupant sensing accuracy is improved, but the challenge of establishing accurate cross-modal associations increases due to multiple occupants
Solution Approach 1:
The patent segments the multi-occupant environment into individual sensor streams and processes each modality's segments separately before association. The AD-TCN model handles multiple wearable sensors and vibration sensors independently, then performs cross-modal association at the segment level, making it feasible to distinguish between multiple occupants
Solution Approach 2:
The patent implements feedback mechanisms where the association model learns from the relationships between vibration and wearable sensor segments. The trained AD-TCN model provides feedback about learned associations that can be used to improve future association decisions, enabling the system to handle multiple occupants more effectively over time
Data Source
AI summary
Cross-modal association (CMA), such as cross-modal signal segment association, is described for associating structural vibration and wearable sensors. This includes an Association Discovery Temporal Convolutional Network (AD-TCN) that determines the amount of shared context between a structural vibration sensor and associated wearable sensor candidates from the parameters of the trained model. CMA achieves improvements in AUC values, F1 scores, and accuracy over relevant baselines. In at least one embodiment, the vibration sensor is associated with a physical structure, the wearable sensor is associated with a person, and the modules estimate association between vibration sensor signals and a person who induces the vibration sensor signals interior to the structure.