Teleoperated Mobile Robot Control for Unpredictable Driving Scenarios
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in handling unpredictable driving scenarios due to the limitations of existing autonomy systems, which may lead to unsafe maneuvers like sudden stops or reversion to a lane, and current teleoperation methods require high operator proficiency and cognitive strain.
Innovation Solution
Implementing a cooperative teleoperation system with an assist scenario database powered by a Large Language Model (LLM) that learns from human input, allowing teleoperators to provide suggestions and validate trajectories, reducing cognitive workload and enhancing decision-making capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If autonomous vehicles use traditional autonomy systems to handle unpredictable driving scenarios, then the system operates autonomously, but it may lead to unsafe maneuvers such as sudden stops or reversion to a lane
Solution Approach 1:
The patent introduces a teleoperator as an intermediary between the autonomous vehicle system and the unpredictable driving scenarios. When the autonomy system encounters scenarios it cannot handle safely, control is transferred to a human teleoperator who can make judgment calls. This mediator approach allows the system to maintain autonomy for predictable scenarios while having human oversight for edge cases, thus improving reliability without sacrificing adaptability.
Solution Approach 2:
The system implements a feedback loop where teleoperator actions and decisions are captured and used to train and improve the autonomy system. The teleoperator's corrections and decisions provide feedback signals that help the autonomous system learn from human expertise, gradually improving its ability to handle unpredictable scenarios safely without requiring constant human intervention.
2Adaptability or versatility
If traditional teleoperation methods are used to handle complex scenarios, then human operators can make decisions, but it requires high operator proficiency and cognitive strain
Solution Approach 1:
Instead of requiring teleoperators to handle all scenarios, the system applies partial action by having the autonomy system handle predictable scenarios independently while only transferring control to the teleoperator when needed for complex or unpredictable scenarios. This reduces the overall cognitive workload on operators while maintaining adaptability for edge cases.
Solution Approach 2:
The autonomy system performs preliminary actions by handling routine and predictable driving scenarios independently before potentially requiring teleoperator intervention. This preliminary autonomous handling reduces the frequency and intensity of teleoperator engagement, thereby reducing cognitive strain while maintaining system adaptability.
3Extent of automation
If full autonomous operation is implemented, then human intervention is minimized, but the system lacks learning capability from human feedback
Solution Approach 1:
The system implements a feedback mechanism where teleoperator decisions and corrections are captured and used to retrain and improve the autonomy system's learning models. This feedback loop allows the system to maintain high levels of automation while continuously learning from human expertise, thus resolving the contradiction between minimizing human intervention and maintaining learning capability.
Solution Approach 2:
The autonomy system performs self-service by automatically learning and improving from the feedback it receives from teleoperator actions. The system uses the captured human expertise to retrain its own models without requiring external intervention, enabling it to maintain high automation levels while continuously adapting and improving its performance.
Data Source
AI summary
An example method to control an autonomous mobile robot includes accessing an assist scenario database, the assist scenario database comprising a large language model(s) related to driving operations and scenarios. The method further includes receiving an autonomous input from the assist scenario database comprising a suggested vehicle trajectory or instruction. The method further includes receiving teleoperator input to authorize or modify the assist scenario database instruction. The method further includes outputting a trajectory signal based on the teleoperator input. The method further includes the autonomous mobile robot executing the trajectory signal or components.