Neural Network Compressing Audio to Haptic Cues for Deaf-Blind Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Haptic communication between users at different locations is not feasible, and individuals with dual-sensory impairments, such as deaf-blindness, face difficulties in communication through audio and visual means, especially when relying on social networking systems that use visual and audio cues.
Innovation Solution
A haptic communication system using cutaneous actuators to transmit haptic signals, which generate haptic forces, vibrations, or motions on a user's skin, allowing for the creation of communication messages through mechanical stimulation, and employing signal processing to enhance communication by constructive or destructive interference between haptic outputs, encoding speech signals using frequency decomposition, and utilizing a neural network to compress audio signals into haptic cues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If audio and visual communication means are used, then communication effectiveness is improved for able-bodied users, but accessibility deteriorates for individuals with dual-sensory impairments
Solution Approach 1:
The patent introduces haptic actuators as an intermediary communication channel that translates audio signals into tactile sensations. This mediator allows information to be transmitted through the sense of touch, enabling deaf-blind users to access communication content without relying on audio or visual senses, thereby resolving the contradiction between communication effectiveness and accessibility.
Solution Approach 2:
The patent replaces acoustic and optical communication systems with a mechanical haptic system. By substituting the traditional audio-visual transmission mechanism with direct mechanical stimulation of the skin, the system enables accessible communication for users with sensory impairments while maintaining effective information transfer.
2Measurement precision
If haptic actuators are placed close together, then spatial resolution of haptic output is improved, but vibration interference between actuators increases
Solution Approach 1:
The patent employs periodic activation patterns where actuators are stimulated in alternating sequences rather than simultaneously. By timing the activation periods of adjacent actuators differently, the system maintains close spacing for high spatial resolution while preventing constructive vibration interference through temporal separation of excitation cycles.
Solution Approach 2:
The system dynamically adjusts the operating parameters of individual actuators based on their spatial relationships. By varying frequency, amplitude, or phase of activation for each actuator according to its position relative to neighbors, the system optimizes both spatial resolution and minimizes interference through real-time parameter modulation.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables effective haptic communication across distances, providing a tactile means of conveying messages and enhancing communication for individuals with sensory impairments by translating audio signals into haptic sensations, thereby improving accessibility and usability.
Implementation Method 1
mechanical stimulation may be used to assist in the creation of communication messages that correspond to a touch lexicon
Implementation Method 2
The cutaneous actuators, spaced apart from each other on a patch of skin, generates haptic outputs by the cutaneous actuators such that the generated haptic outputs constructively or destructively interfere on the patch of skin
Data Source
AI summary
A method comprises inputting an audio signal into a machine learning circuit to compress the audio signal into a sequence of actuator signals. The machine learning circuit being trained by: receiving a training set of acoustic signals and pre-processing the training set of acoustic signals into pre-processed audio data. The pre-processed audio data including at least a spectrogram. The training further includes training the machine learning circuit using the pre-processed audio data. The neural network has a cost function based on a reconstruction error and a plurality of constraints. The machine learning circuit generates a sequence of haptic cues corresponding to the audio input. The sequence of haptic cues is transmitted to a plurality of cutaneous actuators to generate a sequence of haptic outputs.