Sign Language Interpretation Overlay for Media Accessibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current television programs and media lack live sign language interpretation, making it difficult for the hearing impaired and individuals with disabilities to understand and enjoy these media forms, as they may not be able to read closed captions effectively.
Innovation Solution
A system that provides Sign Language Interpretation (SLI) mode, which includes a server, communication network, receiving device, and presentation device, capable of capturing and processing media streams to overlay sign language videos with audio-visual content, using a sign language interpretation library to match text streams with corresponding sign language videos and display them simultaneously.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If closed captioning is provided, then text information is available for hearing impaired viewers, but many young hearing disabled children cannot read or find reading challenging due to disorders such as dyslexia
Solution Approach 1:
The patent introduces sign language interpretation as an intermediary between the audio content and hearing impaired viewers. Instead of requiring viewers to read text captions, the system provides visual sign language avatars that directly convey the audio content through hand gestures and body movements, eliminating the reading barrier while maintaining accessibility
Solution Approach 2:
The system creates visual copies of the audio content in the form of sign language interpretations. The audio stream is translated into corresponding sign language video streams that replicate the meaning and emotion of the original audio, allowing hearing impaired viewers to consume content through a familiar visual language rather than text
2Ease of operation
If sign language interpretation is added to media streams, then hearing impaired viewers can understand content naturally, but the system complexity increases due to multiple processing components
Solution Approach 1:
The patent divides the sign language interpretation system into separate functional modules: audio stream extraction, speech-to-text conversion, text-to-sign-language translation, and sign language video generation. Each module operates independently and can be processed by different services, making the complex system modular and manageable while enabling natural viewing experience
Solution Approach 2:
The system embeds multiple processing layers within each other - the sign language interpretation process is nested within the media stream processing pipeline. The audio stream is processed to generate text, which is then processed to generate sign language videos that are overlaid on the original content, creating a nested structure where multiple functions coexist
3Adaptability or versatility
If sign language videos are overlaid on audio-visual content, then hearing impaired viewers can access information, but the display screen area for original content is reduced
Solution Approach 1:
The patent applies different display qualities to different regions of the screen. The sign language interpretation is displayed in a designated overlay region with enhanced visual characteristics (such as border framing and positioning) that distinguish it from the main content area, allowing both elements to coexist without compromising the original content's visual quality while maintaining accessibility
Data Source
AI summary
Techniques are described by which set-top boxes receive closed-captioning data streams as input to a Sign Language Interpretation (SLI) library. Depending on the demographics, different SLIs are provided. Additionally, input audio stems, e.g., for video programs without closed captioning, are sent to a speech-to-text processor before the SLI library. The text stream is then converted into sign language view mode in a PIP window for single view mode or to a multiview window for dual view mode. The current accessibility setup menu holds the ‘SLI’ option on/off button. SLI library contains videos for vocabulary which are sequenced in the SLI mode view window based on input text from closed captioning stream. If there is a word without a matching video in the SLI library, then the word itself is displayed in the SLI window. Such words are reported to a server for possible future package release with the additions.