Real-Time Text Overlay for Multitasking Meeting Accessibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing collaborative systems do not provide an equitable experience for deaf or hard-of-hearing (D/HH) users, as they lack features that highlight their presence and enable effective text communication during meetings, especially on small screen devices and in situations where speaking is not safe or not possible.
Innovation Solution
A system that integrates real-time text (RTT) with live video and audio streams, chat messages, and live captions, providing graphical highlights and automatic display of RTT even when the user minimizes the meeting window, and allowing for live caption integration with RTT, ensuring all participants can communicate effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If existing systems provide video and audio streams with user highlighting features, then vocal participants can communicate effectively, but deaf or hard-of-hearing participants cannot send messages to presenters when the display area is maximized
Solution Approach 1:
The system separates text communication functionality into a distinct, dedicated overlay interface that operates independently from the main video conference interface. This segmented approach allows D/HH participants to access messaging capabilities without interfering with the presenter's full-screen content while maintaining clear functional boundaries.
Solution Approach 2:
The text communication interface is implemented as a spatial overlay that occupies a different dimensional layer than the main video content. By positioning the text input and display areas in specific screen regions that coexist with maximized video content, the system enables simultaneous access to both presentation material and messaging capabilities.
2Adaptability or versatility
If the meeting window is minimized for multitasking, then users can perform other tasks, but RTT display is lost
Solution Approach 1:
The system pre-configures the RTT overlay interface to automatically persist across window state changes. By establishing the overlay as a persistent layer before the meeting window is minimized, the system ensures that text communication capability remains available even when the main meeting interface is reduced or moved to another workspace.
Solution Approach 2:
The RTT overlay serves as an intermediary interface that bridges the gap between the main meeting window and the desktop environment. This intermediate layer maintains text communication functionality independently of the meeting window's state, allowing users to switch between tasks while preserving communication capabilities.
3Reliability
If graphical highlights are added to show active RTT contributors, then equitable presence is provided, but more processing resources are required
Solution Approach 1:
The system applies graphical highlight indicators selectively only to the specific region of the interface where RTT input is actively occurring, rather than applying effects throughout the entire interface. This localized approach provides visual feedback to D/HH participants about their communication status while minimizing unnecessary processing of the broader interface elements.
Solution Approach 2:
The highlight graphical elements are implemented with moderate visual intensity rather than maximum effect, providing sufficient indication of active RTT contribution without excessive processing overhead. The system applies just enough visual emphasis to ensure equitable presence is achieved while avoiding resource-intensive graphical effects.
4Area of stationary object
If small screen devices are used, then portability is improved, but the ability to display additional UI features for D/HH participants is reduced
Solution Approach 1:
The RTT overlay interface is designed with dynamic layout capabilities that automatically adapt to different screen sizes and resolutions. On smaller devices, the interface elements resize and reposition themselves to maximize available space while maintaining usability, allowing text communication functionality to remain accessible regardless of device form factor.
Solution Approach 2:
The system modifies interface parameters such as font size, input field dimensions, and overlay positioning based on the detected screen characteristics. By dynamically adjusting these parameters for small-screen devices, the system maintains text communication accessibility while optimizing the use of limited display real estate.
Data Source
AI summary
The techniques disclosed herein provide enhanced controls for the display of real-time text (RTT) in calls and meetings. RTT is the ability for someone to send a text message on a character-by-character basis to everybody else in a call or meeting. The system disclosed herein integrates RTT, video, and live captions all in one central experience. This integrated experience allows users to participate equitably by making RTT accessible to users regardless of the operating mode they are in and still concurrently access other meeting content, including video streams, chat messages, live captions, transcripts, and artificial intelligence (AI) tools, such as Copilot. In one embodiment, during an online conference, in response to one of the attendees activating a RTT mode, when at least one user minimizes a meeting stage, such as for the purpose of multitasking while listening, the conference application maintains a display area for displaying RTT.