Deictic Reference Teaching in Conversational Control Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems lack the ability to effectively support human communication modalities, such as pointing gestures, which are essential for human language and interaction, particularly in vehicles and robots, limiting their capability to understand and execute tasks based on deictic references.
Innovation Solution
A conversational command and control system with a deictic reference tool that includes a verification module, decoder module, object recognition module, and haptic module, allowing users to teach and correct deictic references through graphical, spoken, and physical interactions, enabling the system to learn and execute actions on referent objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a conversational command and control system is implemented in a vehicle or machine, then the system can process spoken commands and perform tasks, but it lacks the ability to understand human communication modalities such as pointing gestures and deictic references
Solution Approach 1:
The system is divided into distinct functional modules: a decoder module for interpreting deictic references, an object recognition module for identifying referent objects, and a verification module for confirming understanding. This segmentation allows the system to progressively integrate human communication modalities without overwhelming complexity, addressing each communication aspect through specialized components.
Solution Approach 2:
A deictic reference tool is introduced as an intermediary component that bridges the gap between human pointing gestures and machine command processing. This intermediary translates natural human communication modalities into machine-understandable commands, enabling the system to understand gestures without requiring complete redesign of the core conversational architecture.
2Ease of operation
If the system attempts to learn and execute tasks based on deictic references, then it can perform actions on specific objects, but it requires multiple modules including verification, decoding, and object recognition which increases complexity
Solution Approach 1:
The verification module serves multiple functions: it confirms the system's understanding of deictic references, detects errors in object identification, and coordinates with both the decoder and object recognition modules. This multi-functionality reduces the need for separate specialized components, maintaining ease of operation while managing system complexity through versatile module design.
Solution Approach 2:
The decoder module and object recognition module are integrated into a cohesive deictic reference tool that processes gestures and spoken commands together. By merging these functions into a unified tool rather than separate independent systems, the patent reduces overall complexity while maintaining the capability to learn and execute tasks based on deictic references.
3Reliability
If the verification module confirms understanding of learned objects, then errors can be detected and corrected, but the system requires additional interaction steps for correction
Solution Approach 1:
The verification module implements a feedback mechanism that continuously monitors and confirms the system's understanding of learned objects. When errors are detected in object identification, the system provides feedback to the user and automatically initiates correction procedures, improving reliability through iterative verification rather than requiring extensive manual correction steps.
Solution Approach 2:
The system performs self-verification of its understanding through the verification module, which automatically detects errors in object identification and initiates correction processes without requiring extensive user intervention. This self-service capability improves reliability by maintaining continuous monitoring and correction, reducing the time loss associated with manual error detection and correction.
Data Source
AI summary
A method and system for teaching an object of a deictic reference to a machine. A processor of the machine teaches the object of the deictic reference to the machine which results in the machine learning the object. The teaching includes: the processor finds an item in a region indicated by a physical pointing gesture, by the user, that points to the object; the processor shines a laser light on the item and in response, the processor receives a negative spoken indication from the user that the item shined on by the laser light is not the object; in response to the negative spoken indication from the user, the processor interacts with the user in an iterative procedure wherein the machine learns the object in a final iteration of the procedure. The processor stores the learned object in a storage repository.