Interactive Storytelling System with Dynamic Video Audio Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio and video storytelling technologies, such as audio books and machines, lack interactivity and fail to provide the variation in graphics, roles, and tones that parent-child interactions offer, limiting engagement and interaction.
Innovation Solution
A device with a sharing system, memory, processor, microphone, and display that creates, receives, and matches user inputs with stored stories, allowing for dynamic video and audio data presentation by user-defined roles, voice sources, and image updates based on keyword matching and user inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If audio books or storytelling machines are used to present stories, then the presentation can be automated and consistent, but interactivity and variation in graphics, roles, and tones are lost
Solution Approach 1:
The system captures user inputs through a microphone and uses speech recognition to identify keywords. These keywords trigger dynamic updates to the story presentation, including changing video frames, audio narration, and graphical elements. This feedback loop enables automated systems to respond adaptively to user inputs, resolving the contradiction between automation and interactivity.
Solution Approach 2:
The story presentation is made dynamic through real-time updates based on user inputs. The system can change video frames, audio tracks, and graphical elements during story delivery. This dynamic adaptation allows the automated system to vary the presentation according to user responses, maintaining both automation and versatility.
2Adaptability or versatility
If parents read stories to children manually, then interactivity and parent-child bonding are enhanced, but consistency and time investment increase
Solution Approach 1:
The system enables children to interact with the story independently by capturing their spoken inputs and automatically generating appropriate responses. This self-service capability allows children to explore and engage with the story content without requiring constant parental involvement, reducing the time investment while maintaining interactivity.
Solution Approach 2:
The automated storytelling system acts as an intermediary between parents and children during story time. It facilitates parent-child bonding by creating a shared interactive experience where children can actively participate through speech inputs, while parents can observe and guide without needing to read the entire story themselves.
3Reliability
If static video frames are used in story presentation, then the system is simpler and more reliable, but engagement and vividness are reduced
Solution Approach 1:
The system transitions from static to dynamic video frame presentation by automatically updating frames based on user inputs and story progression. This dynamic updating maintains system reliability through automated control while significantly enhancing engagement and vividness through varied visual content that responds to child interactions.
Data Source
AI summary
A method for presenting video and audio data in an interactive manner which can be applied to story-telling. A story from a book or other source includes preset sentences, each having keywords and each of which correspond to a preset video frame. The keywords represent roles and non-roles, each role corresponding to an image. Voice sources of the roles can be set according to a user instruction. Audio data of storytelling is matched with the keywords of the preset sentences, wherein the audio data is acquired from the voice sources. A matched sentence is acquired from the preset sentences. The preset video frame corresponding to the matched sentence is selected from the database and displayed for interest and vividness of presentation.