AR Negotiation Interface for Distant Human-Robot Collaboration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human-robot collaboration systems lack effective bidirectional communication, particularly in spatially distant environments, limiting their ability to collaborate efficiently and safely, as they often require proximity and are unidirectional.
Innovation Solution
An augmented reality-driven, negotiation-based framework that enables humans and robots to communicate preferences and constraints through an AR interface, allowing visualization of robot plans and incorporation of human feedback for adaptive task planning, facilitating bi-directional communication and collaborative task execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional human-robot collaboration systems are used, then robots can perform tasks autonomously, but effective bidirectional communication between humans and robots is lacking, particularly in spatially distant environments
Solution Approach 1:
The patent introduces an augmented reality interface as an intermediary between humans and robots. This AR interface visualizes robot plans, sensor data, and task progress in the human's field of view, enabling effective communication without requiring physical proximity. The AR overlay acts as a mediator that bridges the spatial gap and facilitates bidirectional understanding.
Solution Approach 2:
The patent replaces traditional mechanical communication methods (proximity-based visual displays, physical handovers) with optical/digital information transmission through AR. Instead of requiring humans to be physically near robots to understand their status, the system uses augmented reality to project robot information into the human's visual field, substituting mechanical proximity requirements with optical information delivery.
2Reliability
If proximity-based robot control systems are used, then communication between humans and robots is possible, but the systems require human presence in close proximity to the robot, limiting operational flexibility
Solution Approach 1:
The AR interface serves as a mediator that decouples communication reliability from physical proximity. By projecting robot status, plans, and sensor information into the human's field of view, the system maintains reliable communication channels regardless of spatial separation, enabling humans to monitor and interact with robots from distant locations.
Solution Approach 2:
The patent adds a visual dimension to robot-human communication by overlaying digital information in the physical workspace. Instead of relying on one-dimensional audio or text communication, the system utilizes the visual dimension through AR overlays, allowing humans to perceive robot status and provide feedback spatially distributed throughout the environment, thereby increasing operational flexibility.
3Loss of information
If unidirectional robot-to-human information display is used, then robots can convey their status and plans, but human feedback cannot be effectively incorporated into robot planning
Solution Approach 1:
The patent implements a feedback loop where human interactions with AR overlays (such as selecting objects, modifying task parameters, or providing corrections) are captured and transmitted back to the robot's planning system. This feedback mechanism enables the robot to adjust its plans based on human preferences, constraints, or corrections, transforming unidirectional information display into bidirectional collaborative planning.
Solution Approach 2:
The AR interface serves multiple functions simultaneously: it displays robot status and plans, captures human feedback, transmits constraints and preferences, and visualizes updated plans. This multi-functional design consolidates what would otherwise require separate systems into a single integrated interface, reducing overall system complexity while enabling bidirectional communication.
4Loss of information
If projection-based spatial information systems are used, then robots can show their planned trajectories and motion intent, but the human must be in close proximity to the robot and bidirectional communication is difficult
Solution Approach 1:
The AR interface acts as an intermediary that captures and projects robot spatial information (trajectories, motion intent, sensor data) directly into the human's field of view. Instead of requiring projection onto physical surfaces near the robot, the system uses AR to overlay spatial information in the human's visual space, eliminating the proximity requirement while maintaining accurate spatial representation.
Solution Approach 2:
The patent transitions from two-dimensional projection surfaces to three-dimensional spatial overlay in the human's field of view. By rendering robot trajectories and spatial information as AR overlays that maintain proper spatial relationships and perspectives, the system provides intuitive spatial understanding without requiring the human to be physically near the robot or interact with projection surfaces.
Data Source
AI summary
Effective human-robot collaboration (HRC) requires extensive communication among the human and robot teammates, because their actions can potentially produce conflicts, synergies, or both. An augmented reality-driven, negotiation-based (ARN) framework is provided for HRC, where ARN supports planning-phase negotiations within human-robot teams. Experiments in an office environment, where multiple mobile robots work on delivery tasks, where the robots could not complete the tasks on their own, but sometimes need help from their human teammate, making human-robot collaboration necessary. In comparison to a non-AR baseline, ARN significantly improved the human users' work efficiency, and their cognitive load, while reducing the overall task completion time of the whole team.