Virtual Control Interface Overlay for Video Call Remote Assistance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality technologies for remote assistance are inadequate in providing effective guidance for users, especially those with vision and hearing impairments, as they rely on video calls which can be complex and difficult to understand, especially when operating household appliances.
Innovation Solution
A remote assistance method and system that uses a video call process to send device information to a server, virtualize an operation interface, and display it on the video call screen, allowing users to perform operations intuitively and synchronizing visual effects to assist in controlling the target device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video call is used for remote assistance, then real-time communication is achieved, but operation guidance becomes complex and difficult to understand
Solution Approach 1:
The patent creates a virtual copy of the target device's operation interface and displays it within the video call screen. This virtual interface replicates the actual device controls, allowing the assistant to manipulate the copied interface while the user observes the corresponding real-device operations through the video stream, thereby simplifying guidance while maintaining real-time communication
Solution Approach 2:
The patent merges the video call interface with the virtual operation interface into a single integrated display. The virtual control panel is overlaid on top of the video stream, combining communication and control functions in one unified interface, eliminating the need for users to switch between separate applications or windows
2Adaptability or versatility
If traditional video call guidance is used, then remote assistance is provided, but effectiveness is reduced for users with vision and hearing impairments
Solution Approach 1:
The patent replaces audio-based verbal instructions with visual-based graphical interface manipulation. Instead of relying on hearing-impaired users to understand spoken guidance, the system uses visual demonstration where the assistant interacts with a virtual interface that mirrors the real device, making the guidance accessible to users with hearing impairments while maintaining effectiveness
3Ease of operation
If virtual operation interface is displayed on video call screen, then operation guidance becomes intuitive, but device complexity increases
Solution Approach 1:
The patent introduces a server as an intermediary that handles the complex tasks of capturing the target device screen, generating the virtual operation interface, and transmitting it to the terminal. This mediator approach allows the terminal device to display a sophisticated virtual interface without requiring complex local processing capabilities, distributing system complexity across multiple components
Data Source
AI summary
A method, performed by a first terminal in a video call process between the first terminal and a second terminal, includes sending an image including a target device to a server, receiving device information corresponding to the target device sent by the server, virtualizing an operation interface based on the received device information, and displaying the operation interface on a current video call screen, where the operation interface is an operation interface of a control panel or a remote control of the target device, and displaying, on the current video call screen based on data from the second terminal, one or more operations performed on the operation interface by a user of the second terminal on the second terminal.