Multimodal Notification System for Hands-Free Data Entry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In speech-assisted work environments, especially those with less predictable and less repetitive workflows, workers face difficulties in efficiently using speak-ahead features due to unfamiliarity with upcoming prompts, leading to reduced efficiency and challenges for inexperienced workers.
Innovation Solution
A multimodal system that provides visual and audible notifications indicating valid input modes for each prompt, allowing workers to easily determine the appropriate input components and modes for responding to prompts, and enhances the speak-ahead feature by displaying the sequence of upcoming prompts for efficient data entry.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If workers rely on speech prompts in unpredictable workflows, then hands-free operation efficiency is maintained, but worker efficiency decreases due to inability to anticipate upcoming prompts
Solution Approach 1:
The patent introduces a visual display dimension to complement the auditory speech prompts. The mobile device displays upcoming prompts and accepted input modes on its screen, allowing workers to see what information will be requested and how it can be provided. This additional visual dimension enables workers to anticipate prompts and prepare responses without compromising hands-free operation, thereby resolving the contradiction between ease of operation and productivity.
2Productivity
If speak-ahead feature is used without prompt sequence display, then data entry speed increases, but inexperienced workers cannot utilize the feature in infrequent tasks
Solution Approach 1:
The system displays the sequence of upcoming prompts before they are verbally presented. This preliminary visual information allows workers to mentally prepare and organize their responses in advance. For inexperienced workers performing infrequent tasks, this advance knowledge of the prompt sequence enables them to effectively use the speak-ahead feature, as they can see what information will be requested and prepare appropriate responses, thereby improving both productivity and adaptability.
3Adaptability or versatility
If multiple input modes are available for each prompt, then system versatility increases, but worker confusion increases without clear indication of valid modes
Solution Approach 1:
The patent applies local quality by providing specific, prompt-specific information about accepted input modes. Rather than presenting all possible input modes universally, the system displays only the valid input modes for each specific prompt (e.g., showing that a barcode scanner is accepted for product identification but not for quantity). This localized information approach maintains system versatility while eliminating worker confusion by clearly indicating which modes are appropriate for each specific input request.
Data Source
Figure 1~3
Figure 2
Figure 4A~4B
AI summary
A system for executing a multimodal software application 52 includes a mobile computer device 14 with a plurality of input interface components, the multimodal software application 52, and a dialog engine 62 in operative communication with the multimodal software application 52. The multimodal software application 52 is configured to receive first data from the plurality of input interface components. The dialog engine 62 executes a workflow description 60 from the multimodal software application 52 by providing prompts to an output interface component. Each of these prompts includes notification indicating which of the input interface components are valid receivers for that respective prompt. Furthermore, the notification may indicate the current prompt and at least the next prompt in sequence.