Structured Sound Records for DHH Audio Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
People who are deaf or hard of hearing (DHH) face difficulties in perceiving and understanding various sounds, especially in environments without hearing assistive devices, as they may not receive adequate auditory cues.
Innovation Solution
A computing device generates structured sound records with descriptive labels and timestamps, which can be used to produce non-audio indications such as haptic alerts, graphical displays, or light signals, allowing DHH individuals to be alerted to important sounds through connected devices, and provides a timeline representation of sounds for better understanding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If auditory information is provided to DHH individuals, then they can perceive sounds, but they cannot understand or detect sounds without hearing assistive devices
Solution Approach 1:
The patent introduces an intermediary system that captures audio through microphones, processes it through machine learning models to identify sound sources and characteristics, and presents the information through alternative modalities (visual displays, haptic feedback). This intermediary translates auditory information into forms accessible to DHH individuals without requiring complex hearing assistive devices
Solution Approach 2:
The patent replaces the mechanical/acoustic hearing system with an electronic processing system. Instead of relying on physical hearing mechanisms, the system uses microphones to capture sound waves, converts them to electrical signals, processes them through algorithms, and outputs visual or haptic representations, thereby substituting the biological hearing mechanism with an electronic information processing chain
2Reliability
If traditional audio alerts are used, then emergency notifications are provided, but DHH individuals cannot detect them
Solution Approach 1:
The patent transitions emergency alerts from the auditory dimension to visual and haptic dimensions. Instead of relying solely on sound waves, the system displays visual indicators on screens and generates haptic patterns through vibration motors, adding spatial and tactile dimensions to emergency notification delivery
Solution Approach 2:
The patent changes the physical parameters of alert delivery by converting acoustic pressure waves into electromagnetic displays and mechanical vibrations. The system monitors audio parameters, identifies emergency patterns, and transforms them into different physical manifestations (light intensity, vibration frequency) that are detectable by DHH individuals
3Loss of information
If audio data is processed to generate structured sound records, then sound information becomes accessible, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary action by continuously monitoring and processing audio data in the background, pre-identifying sound sources and characteristics before user interaction is needed. The machine learning models are trained in advance and ready to rapidly classify sounds when audio input is received, reducing real-time processing delays
Solution Approach 2:
The patent segments the audio processing task into distinct stages: audio capture, signal processing, machine learning classification, and output generation. This segmentation allows parallel processing of different audio streams and enables the system to handle multiple sound sources independently, improving overall processing efficiency
Data Source
AI summary
An example method includes receiving, by one or more processors of a computing device, audio data recorded by one or more microphones of the computing device; and generating, based on the audio data and by the one or more processors, one or more structured sound records, a first structured sound record of the one or more structured sound records including: a description of a first sound, the description including a descriptive label of the first sound, the descriptive label different than a text transcription of the first sound, and a time stamp indicating a time at which the first sound occurred; and outputting a graphical user interface including timeline representation of the one or more structured sound records.