Cross-Device Task Guidance via Shared State and Modality Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face inefficiencies in providing task guidance, as they often require users to view displays with their hands occupied or move away from devices, and audio-based guidance struggles with skipping instructions, communicating complex concepts, and long sequences of steps.
Innovation Solution
The implementation of computerized task guidance across multiple devices and applications using a shared digital task document and state data structure, allowing seamless interaction between visual and natural language modalities, enabling hands-free/eyes-free access and dynamic transformation of instructions for different interaction modalities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If visual display-based guidance is used, then detailed task instructions can be shown, but users must view displays with hands occupied or move away from devices
Solution Approach 1:
The system segments task guidance into multiple modalities: visual display for detailed instructions and audio output for hands-free guidance. The digital working task document is divided into actionable items that can be presented through different channels, allowing users to access information without occupying hands or moving from the device.
Solution Approach 2:
The computing device is designed to provide multiple user interface modalities simultaneously - visual display, audio output, and natural language dialog. This multi-functional approach allows the same device to serve different user needs and contexts, enabling hands-free operation while maintaining detailed instruction accessibility.
2Ease of operation
If audio-based guidance is used, then hands-free access is enabled, but the system struggles with skipping instructions and communicating complex concepts
Solution Approach 1:
The system dynamically adapts the presentation of task instructions based on user interactions and context. The digital working task document can be dynamically updated with format changes to suit different interaction modalities, allowing flexible navigation and skipping of instructions while maintaining hands-free operation capability.
Solution Approach 2:
A digital working task document serves as an intermediary between the base task document and the user interactions. This intermediary structure enables sophisticated processing of user commands, allowing the system to skip instructions and communicate complex concepts effectively while maintaining hands-free operation.
3Adaptability or versatility
If multiple devices with different user interface modalities are used, then comprehensive task guidance can be provided, but system complexity increases
Solution Approach 1:
The system merges multiple user interface modalities (visual display, audio output, natural language dialog) into a unified task guidance system. By combining these modalities through a shared digital working task document and task state data structure, the system provides comprehensive guidance while managing complexity through integration rather than separate independent systems.
Solution Approach 2:
The system creates a digital working task document that is a dynamic copy and transformation of the base task document. This working document can be accessed and updated across multiple devices, providing a consistent state representation that simplifies cross-device integration while supporting multiple interaction modalities.
4Stability of the object's composition
If state is maintained across multiple devices, then consistent task guidance experience is achieved, but data synchronization complexity increases
Solution Approach 1:
A task state data structure serves as an intermediary for maintaining and synchronizing task state across multiple devices. This centralized state representation ensures consistency in task guidance experience while simplifying synchronization by providing a single source of truth that can be accessed and updated across devices without complex synchronization logic.
Data Source
AI summary
A digital task document can include instructions for performing a task, and a task state data structure can indicate a state of completion of the task. A first update of the data structure can be performed in response to visual user input received from a user profile via a first computer application/device. A second update of the data structure can be performed in response to natural language input received from the user profile via the second computer application/device. A first set of task guidance can be provided to the user profile via the first application/device in a visual format by displaying the task document on a computer display. A second set of task guidance can be provided to the user profile via the second application/device in a natural language format. The first and second sets of task guidance can be provided using the task document and the data structure.