Voice Content Highlighting for Hands-Free Audio Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Individuals face challenges in marking portions of pre-recorded digital audio/video content for later access while engaged in activities that prevent manual device interaction, such as driving or housework, and host-specified clickable timestamps are not feasible in these situations.
Innovation Solution
An AI-based computing tool uses machine learning models to identify content portions of interest based on spoken user queries, converting audio to textual representations and storing them for later retrieval without requiring physical device manipulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If clickable timestamps are provided by the host, then users can link directly to specific timeslots, but users still require manual physical interaction to select timestamps which is not feasible when hands are occupied
Solution Approach 1:
The patent replaces manual mechanical interaction (clicking timestamps with fingers) with voice-based acoustic interaction. Users can speak commands like 'highlight this section' while driving, and the system uses speech-to-text conversion and natural language processing to automatically identify and highlight the desired content portion, eliminating the need for hand manipulation of the device.
Solution Approach 2:
The system enables users to independently highlight content through voice commands without requiring host-provided timestamps or manual device operation. The automated system listens to the audio content, processes user spoken queries, and autonomously identifies and marks the relevant portions, making the user self-sufficient in content navigation.
2Reliability
If users wait until later to manually highlight content, then they can operate the device, but they may have forgotten which content they wanted to access
Solution Approach 1:
The system performs content highlighting in real-time during playback rather than requiring deferred action. As the audio content plays and users listen, they can immediately issue voice commands to highlight sections of interest, and the system processes these commands on-the-fly to mark the content for later retrieval, eliminating the time delay and memory loss associated with waiting.
3Adaptability or versatility
If users manually manipulate the device to highlight content, then they can identify portions of interest, but this is unsafe and impractical during activities like driving
Solution Approach 1:
The patent substitutes manual mechanical device manipulation with voice-based acoustic commands. Users can safely speak highlighting commands while driving or performing other activities, and the system captures these voice inputs through the device's microphone, processes them via speech-to-text conversion, and executes the content highlighting without requiring the user's hands to leave their current safe positions.
Data Source
AI summary
A machine learning (ML) computer model is trained on a dual-modal input sequence comprising target content data and an input user query specifying attributes for locating a portion within the target content data. The ML computer model is trained to process the dual-modal input sequence and output an output text specifying start and end indices, within the target content data, corresponding to the portion. Runtime content data, and a user spoken query, are received and the runtime content data and user spoken query are processed via the trained ML computer model to generate an output text specifying a predicted beginning index and predicted ending index within the runtime content data. The portion of the runtime content data corresponding to a range of runtime content data bounded by the beginning index and ending index, inclusively, is retrieved and stored for later retrieval and processing.