VR Robotic Control via Virtual World State Differences
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robotic control systems are limited in adapting to uncertain, unfamiliar, or changing environments, and require prehandling of potential environmental inputs, leading to inefficiencies in data collection and task execution, especially in complex tasks like sorting recyclables from municipal solid waste.
Innovation Solution
A system that allows humans to interact with virtual representations of real-world objects in a virtual reality environment, where the differences in the virtual world are translated into inputs for a robotic actuator to autonomously manipulate real-world objects, enabling efficient data collection and task execution through a game-like interface that engages users and provides real-time feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated control with preprogrammed responses is used, then the robot can perform repetitive tasks efficiently, but it cannot adapt to uncertain, unfamiliar, or changing environments
Solution Approach 1:
The system performs preliminary actions by having users interact with virtual representations of objects before the robot executes physical manipulation. Users pre-handle potential environmental inputs by manipulating virtual objects, which generates preprocessed instructions that guide the robot's subsequent actions in the physical world.
Solution Approach 2:
A virtual world intermediary system is introduced between the user and the physical robot. This virtual environment serves as a mediator that translates user interactions into robot commands, allowing users to interact with virtual representations of real-world objects rather than directly controlling the physical robot.
2Adaptability or versatility
If remote control with human operator input is used, then the robot can adapt to changing environments, but it requires continuous human involvement and cannot operate autonomously
Solution Approach 1:
The system creates virtual copies of real-world objects and environments. Users interact with these virtual representations rather than directly controlling the physical robot. The virtual world serves as a simplified model that captures essential features of the physical environment, enabling autonomous operation based on virtual interactions.
Solution Approach 2:
Direct mechanical control of the robot by human operators is replaced with an informational system. User interactions with the virtual environment generate data and instructions that automatically guide the robot, substituting continuous mechanical control with automated instruction execution based on virtual manipulations.
3Ease of operation
If direct virtual reality control is used, then user engagement and data collection improve, but the system still relies on continuous human operator input in real time
Solution Approach 1:
Users perform preliminary interactions with virtual objects to generate instructions that the robot executes autonomously. The user's engagement in manipulating virtual representations creates the necessary control data in advance, eliminating the need for continuous real-time human input during robot operation.
Data Source
AI summary
A robotic system and method performs tasks autonomously on real-world objects (RWOs) upon receipt of inputs from a virtual reality environment. A robotic actuator has tools to manipulate the RWOs, and a set of robot-specific instructions for actuating the tools. A first set of rules governs the RWOs, and sensors detect the presence of the RWOs. A virtual reality (VR) computer uses sensor data to generate a virtual world (VW) including virtual world objects (VWOs) representative of RWOs. A second set of rules governs the VWOs. A user manipulates the VWOs to generate a modified virtual world (MVW). A transformation engine captures differences between the VW and MVW, transforms the differences into inputs for the robotic actuator. An autonomous instruction engine receives the inputs to identify RWOs and combines the inputs with the first set of rules to generate combined instructions usable by the robotic actuator to manipulate RWOs autonomously.