Multimodal Command Detection via Sliding Window Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing human-computer interaction systems face challenges in detecting multimodal commands initiated by user input across multiple modalities, such as visual, auditory, and tactile, due to variations in processing speeds and complexities in correlating input events from different devices.
Innovation Solution
A computing device receives and correlates input events from multiple devices across various modalities, transforming them into modality-independent events or keeping them in their native form, and uses a sliding window approach to determine when a multimodal command is invoked, allowing for the execution of computing operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If input events from multiple modalities are correlated to detect multimodal commands, then the capability to detect multimodal commands is improved, but the device complexity increases
Solution Approach 1:
The patent introduces an intermediary correlation mechanism that mediates between multiple input devices and the command detection logic. This intermediary layer standardizes the correlation process by establishing a unified framework for matching input events across different modalities, thereby reducing the overall system complexity while maintaining enhanced detection capability.
Solution Approach 2:
The patent segments the complex task of multimodal command detection into distinct components: individual input device handlers, event correlation modules, and command invocation logic. This segmentation allows each component to be optimized independently and simplifies the overall system architecture by breaking down the complex correlation process into manageable, modular units.
2Measurement precision
If a sliding window approach is used to determine multimodal command invocation, then the accuracy of command detection is improved, but the processing time increases
Solution Approach 1:
The sliding window approach implements partial action by analyzing only a limited, relevant subset of input events within a defined time window, rather than processing all possible combinations. This selective processing maintains detection accuracy for commands occurring within the window while significantly reducing the computational burden compared to exhaustive analysis of all input sequences.
Solution Approach 2:
The system performs preliminary actions by pre-defining the sliding window parameters and correlation criteria before actual command detection occurs. This preliminary configuration establishes the processing framework in advance, enabling faster real-time detection by avoiding dynamic parameter adjustment during event correlation, thus reducing processing time while maintaining precision.
Data Source
AI summary
Technology for detecting multimodal commands that enhance the human-computer interaction of a computing device. In an illustrative implementation, a computing device may receive multiple input events from a plurality of input devices. The plurality of input devices may each correspond to a different computer input modalities and the computing device may correlate the input events across different modalities. The computing device may keep the input events in their native form (e.g., input device specific) or may transform the input events into modality independent events. In either example, the computing device may determine the events satisfy a definition for a multimodal command that identifies multiple events from different computer input modalities. Responsive to the determination, the computing device may invoke the multimodal command on the client device to perform one or more computing operations.