Server-Side RUI Session Mirroring for Companion Device Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote user interface (RUI) systems fail to effectively synchronize and control multiple devices during a RUI session, leading to a less than desirable user experience as multiple RUIs are presented without proper linking and reconciliation.
Innovation Solution
A server provides a remote user interface (RUI) and video to a main screen, which is discoverable by a companion device, and mirrors the graphical part of the RUI session on both the main screen and the companion device, allowing touch commands to be processed as if they came from the main screen, using HTML5 browser compatibility and transcoding of graphics rendering commands to ensure synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple RUIs are presented on multiple devices without linking, then device functionality is expanded, but session synchronization and control reconciliation deteriorate
Solution Approach 1:
The patent merges multiple RUI sessions across different devices by establishing them as interconnected instances of the same logical session. The server maintains a unified session state that is synchronized across the main display device and companion devices, allowing touch commands from any device to be processed as if they originated from the main screen. This combining approach enables expanded device functionality while maintaining reliable session synchronization through a centralized session management architecture.
2Ease of operation
If RUI graphics are mirrored on companion device with HTML5 browser, then accessibility and touch control are improved, but protocol compatibility and graphics rendering complexity increase
Solution Approach 1:
The patent introduces an intermediary layer in the form of a protocol translation mechanism that converts the proprietary RUI graphics protocol into HTML5-compatible rendering commands. This intermediary translator enables the companion device's HTML5 browser to render RUI graphics without requiring native protocol support, thereby improving ease of operation through universal browser compatibility while managing rendering complexity through standardized web technologies.
3Reliability
If companion device is paired with main screen, then session mirroring is achieved, but discovery and pairing setup time increase
Solution Approach 1:
The patent implements self-service pairing mechanisms where the companion device automatically discovers and connects to the main display device without requiring manual user configuration. The system employs automatic session discovery protocols that enable the companion device to autonomously establish connections and begin mirroring RUI sessions, thereby achieving reliable session mirroring while minimizing pairing setup time through automated device discovery and connection establishment.
Data Source
AI summary
A server sends to a video device such as a TV a remote user interface (RUI) that is presented on the TV and manipulable to send control commands back to the server. A companion device such as a tablet computer discovers the RUI session and is provided by the server with its own RUI, which mirrors that on the TV, modified as appropriate for the screen of the companion device. The server maintains the two RUIs synchronized such that the RUI on the companion mirrors the RUI on the TV.