mmWave Sleep Posture Monitoring With 3D Joint Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mmWave imaging systems face challenges in accurately extracting human body joint information and distinguishing human shape from surrounding objects during sleep due to signal absorption and low resolution, leading to incomplete images and difficulty in inferring joint locations.
Innovation Solution
A software-only sleep posture monitoring system, MiSleep, uses built-in millimeter-wave technology on 5G devices and deep learning models to predict 3D joint locations from mmWave reflections, incorporating a customized Deep Convolutional Neural Network (DCNN) and a Height Classifier to enhance accuracy and generalizability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If traditional mmWave imaging is used for sleep posture monitoring, then the system can work under dark conditions and maintain privacy, but the image resolution is extremely low and many high-frequency components are eliminated
Solution Approach 1:
The patent applies local quality by enhancing specific frequency components of the mmWave signal. Instead of processing the entire signal uniformly, the system identifies and enhances high-frequency components that correspond to body contours and limbs, while maintaining the low-resolution nature of the overall image. This selective enhancement allows the system to work in dark conditions with low overall resolution while still extracting meaningful posture information from localized high-frequency features.
2Ease of operation
If mmWave signals are used for posture monitoring, then privacy is protected and no additional hardware is needed, but signals are absorbed by body parts or specularly reflected away, causing missing parts in the output human shape
Solution Approach 1:
The patent employs feedback mechanisms through deep learning models that iteratively refine the reconstruction of body parts. The system uses the received mmWave signals as input, processes them through neural networks that learn from training data, and generates enhanced images that compensate for missing parts. The feedback loop allows the model to infer missing body contours and limb positions based on patterns learned from numerous training examples, thereby recovering information that would otherwise be lost due to signal absorption and reflection.
3Ease of manufacture
If mmWave transceivers are integrated into 5G devices for widespread deployment, then the system becomes cost-effective and accessible, but the extremely low resolution makes it difficult to separate human shape from bed and surrounding objects
Solution Approach 1:
The patent introduces deep learning models as an intermediary between the raw mmWave signals and the final posture classification. These models serve as a mediator that learns to distinguish human shapes from bed and surrounding objects by training on labeled data. The intermediary processing layer extracts relevant features and suppresses background clutter, enabling the system to achieve accurate posture monitoring despite the low resolution inherent to cost-effective mmWave transceivers.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
MiSleep achieves accurate prediction of 3D body joint locations during sleep, comparable to vision-based systems, without additional hardware, enabling fine-grained, non-invasive posture monitoring under dark conditions.
Implementation Method 1
receiving millimeter-wave (mmWave) wireless signals reflecting from the human subject
Implementation Method 2
mmWave signals could be absorbed by many body parts or specularly reflect from them
Data Source
AI summary
Methodology and corresponding apparatus pertain to human sleep posture monitoring, using a wireless signal-based monitoring system leveraging millimeter-wave technology. A software-only human sleep posture monitoring solution based on millimeter-wave (mmWave) wireless-based solutions enables fine-grained posture monitoring under no light without being privacy-invasive. In zero visibility, body joint information and changes can be extracted directly from mmWave imaging using improved capabilities for extracting human sleep posture data from millimeter-wave wireless systems. A single-person sleep posture monitoring system leverages signal processing and deep learning models to enable fine-grained monitoring continuously and non-intrusively with commodity (i.e., generally available) mmWave devices. The system directly predicts joint locations from reflected mmWave signals by learning the hidden association between them from thousands of data samples. Learning is accomplished through a customized Deep Convolutional Neural Network (DCNN), that predicts the 3D locations of several key body joints from the reflected signals captured by multiple mmWave antennas.