AR Robot Task Authoring With Dynamic SLAM for IoT Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for robot navigation and task planning in IoT environments are limited by fixed camera views, occlusion issues, outdated SLAM maps, and lack of spatial awareness, making it difficult for robots to adapt to changing environments and perform complex tasks in daily settings.
Innovation Solution
An AR-SLAM-based authoring system using a mobile device that allows users to create dynamic SLAM maps and spatially plan robot tasks, enabling robots to navigate and interact with IoT devices in a physically aware and adaptive manner, with features like logic-driven event scheduling and contextual feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If external vision systems with fixed camera views are used for robot tracking, then the authoring interface is simplified, but the authoring scene is limited to a fixed perspective and cannot accommodate mobile or distributed task authoring
Solution Approach 1:
The patent replaces the fixed mechanical camera system with a virtual camera system implemented through AR software. The mobile device's camera captures the physical environment, and the AR interface renders a virtual camera view that can be dynamically positioned and oriented anywhere in the 3D space, substituting physical camera constraints with software-based flexibility.
Solution Approach 2:
The patent transitions from a 2D fixed camera view to a 3D spatially-aware AR environment. By incorporating depth information from the mobile device's sensors and creating a three-dimensional authoring space, users can position virtual elements and define robot tasks from multiple perspectives and locations throughout the environment, not just from a single fixed viewpoint.
2Device complexity
If pre-scanned SLAM maps are used for robot navigation, then the authoring interface is separated from navigation, but the SLAM maps become static and cannot adapt to environmental changes
Solution Approach 1:
The patent implements dynamic SLAM map generation where the map is continuously updated in real-time as the mobile device moves through the environment. Rather than using a static pre-scanned map, the system actively reconstructs and updates the spatial representation of the environment, allowing it to adapt to changes such as moved objects or altered layouts.
Solution Approach 2:
The system performs self-updating of the SLAM map through the mobile device's continuous environmental scanning and processing. The AR interface automatically regenerates the spatial map based on current sensor data without requiring manual re-surveying or external intervention, enabling the navigation system to adapt autonomously to environmental changes.
3Adaptability or versatility
If hand-held or head-mounted AR devices are used for mobile task authoring, then users can move around and author from different perspectives, but the limited field-of-view constrains the robot's navigation range
Solution Approach 1:
The patent makes the mobile device serve multiple functions: it acts as both the authoring interface (capturing images, rendering AR) and the navigation system (generating SLAM maps, calculating robot paths). This multi-functionality allows the system to overcome the limited field-of-view constraint by using the same device data for both authoring and navigation planning, enabling robots to navigate beyond what is currently visible in the AR view.
Solution Approach 2:
The patent introduces a virtual camera system as an intermediary between the physical mobile device and the robot navigation system. The virtual camera, positioned anywhere in the 3D space, captures images that are used to generate SLAM maps and plan robot paths, allowing the robot to navigate to locations outside the mobile device's physical field-of-view while the user remains within their comfortable viewing range.
4Ease of operation
If IoT devices are controlled through mobile interfaces, then access and control are enabled, but the interfaces lack spatial awareness and cannot coordinate distributed devices effectively
Solution Approach 1:
The patent introduces the AR interface as an intermediary layer between the user and IoT devices. This AR interface superimposes virtual representations of IoT devices and their control options directly onto the physical environment, maintaining spatial relationships. Users can see and control distributed IoT devices in their correct spatial contexts, preserving spatial distribution information while providing easy mobile access.
Solution Approach 2:
The patent applies local quality by rendering IoT device controls and information at their specific spatial locations within the AR environment. Each IoT device is represented with its own control interface positioned near its physical location, allowing users to access and control devices contextually based on their spatial distribution rather than through a flat, location-agnostic menu system.
Data Source
AI summary
Disclosed is a visual and spatial programming system for robot navigation and robot-IoT task authoring. Programmable mobile robots serve as binding agents to link stationary IoT devices and perform collaborative tasks. Three key elements of robot task planning (human-robot-IoT) are coherently connected with one single smartphone device. Users can perform visual task authoring in an analogous manner to the real tasks that they would like the robot to perform with using an augmented reality interface. The mobile device mediates interactions between the user, robot(s), and IoT device-oriented tasks, guiding the path planning execution with Simultaneous Localization and Mapping (SLAM) to enable robust room-scale navigation and interactive task authoring.