AR Vehicle Command Interface for Faster Autonomous Fleet Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for remotely monitoring and controlling autonomous vehicles (AVs) are inefficient, requiring users to repeatedly switch between viewing AVs and handheld devices for information and task management, leading to potential errors and decreased productivity.
Innovation Solution
An augmented reality system that allows users to interact with AVs using a headset, presenting graphical user interfaces overlaid on the physical environment, enabling intuitive selection and control of AVs without needing to manually input commands, thereby improving interaction speed and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users use handheld devices to monitor and control AVs, then information can be accessed and commands transmitted, but users must repeatedly switch between viewing AVs and devices, reducing interaction speed and increasing error potential
Solution Approach 1:
The patent introduces an augmented reality interface as an intermediary between the user and the AV fleet management system. This AR interface overlays vehicle information, selection options, and control commands directly in the user's field of view, eliminating the need to switch between physical devices and visual targets. The intermediary AR system processes user gestures and gaze inputs to transmit commands to the AVs, resolving the contradiction by providing both spatial proximity to the AV and digital connectivity to the control system.
Solution Approach 2:
The patent transitions the user interface from a two-dimensional handheld device screen to a three-dimensional augmented reality space that coexists with the physical AV environment. By projecting informational layers into the spatial dimension where the AVs are located, users can access and interact with the fleet management system without breaking their visual focus or physical orientation toward the vehicles, thus eliminating time loss and improving operational ease.
2Reliability
If users manually input commands on handheld devices, then precise control can be achieved, but the process becomes complex and time-consuming
Solution Approach 1:
The augmented reality system enables self-service interaction where the interface automatically adapts to user needs and context. The system detects user gaze direction, gesture intentions, and situational parameters to dynamically present relevant AVs and commands, eliminating the need for users to navigate complex menus or manually input detailed parameters. This self-adjusting interface maintains high command accuracy by using multiple sensing modalities while reducing interaction complexity through automated contextual understanding.
Solution Approach 2:
The patent dynamically changes interface parameters such as information display density, interaction mode, and command presentation based on user behavior and operational context. The system adjusts these parameters in real-time to optimize both accuracy and simplicity, transforming the static complex menu structure into a dynamic adaptive interface that simplifies interactions while maintaining precise control capabilities when needed.
3Loss of information
If traditional remote monitoring systems are used, then AV information can be accessed, but productivity decreases due to inefficient information retrieval and task management
Solution Approach 1:
The patent merges the physical monitoring environment with the digital information layer by overlaying AV data, alerts, and control interfaces directly onto the user's view of the vehicles. This merging eliminates the separation between information access and physical observation, allowing users to retrieve and act on information simultaneously without switching attention or devices, thus improving productivity while maintaining complete information accessibility.
Solution Approach 2:
The augmented reality system performs preliminary actions by pre-processing and pre-presenting relevant information before the user needs it. The system anticipates user needs by detecting contextual cues and proactively displaying pertinent AV information, selection options, and command suggestions in the optimal location and format, reducing the time and cognitive effort required for information retrieval and task management while maintaining comprehensive data access.
Data Source
AI summary
Among other things, techniques are described for monitoring and controlling autonomous vehicles (AVs). As an example, at least one processor determines that a first AV is in a field of view of a user wearing an augmented reality display device, determines first data regarding an operation of the first AV, and causes at least a portion of the first data to be presented to the user using the augmented reality display device. For example, a graphical user interface is presented in the field of view of the user using the augmented reality display device, and at least the portion of the first data is included on the graphical user interface, such that at least the portion of the first data appears to be arranged spatially proximal to the first AV in the field of view of the user.