Semi-Transparent Virtual Keyboard Overlay for VNC Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Virtual Network Computing (VNC) systems, objects such as soft keyboards often disappear or become non-interactive when users are interacting with a local device to view or perform actions on a remote device, leading to difficulties in providing inputs, as they are blocked from device screenshots and screen recordings, preventing users from performing actions like tapping buttons.
Innovation Solution
An Application Programming Interface (API) is used to retrieve the page structure of the remote device in XML type, allowing the creation of a dummy semi-transparent keyboard layout above the VNC component on the local device, which does not respond to user actions, enabling users to know the real state of the soft keyboard and guiding them through the layout without blocking interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a soft keyboard is displayed on the local device during VNC interaction, then users can see the keyboard layout and know what to tap, but the keyboard blocks interactions with elements beneath it on the VNC screen
Solution Approach 1:
The patent introduces a semi-transparent overlay layer between the VNC screen and the user interface. This overlay displays keyboard layout information without fully blocking the underlying VNC elements, allowing users to see both the keyboard layout guidance and the interactive elements beneath it simultaneously. The semi-transparent nature of the overlay acts as an intermediary that provides information while maintaining pass-through interaction capability.
2Ease of operation
If the soft keyboard is made transparent to allow interactions, then users can tap buttons through it, but users cannot see the keyboard layout and guidance
Solution Approach 1:
The patent employs transparency and opacity adjustments as a visual property modification strategy. By controlling the transparency level of the keyboard overlay, the system dynamically balances visibility and interactivity. The keyboard layout is rendered with partial transparency, allowing users to perceive the layout structure and guidance text while still being able to interact with elements beneath the overlay through touch or click inputs.
3Manufacturing precision
If objects like soft keyboards are blocked from device screenshots, then screen recording quality is improved, but users cannot perform actions like tapping buttons that require visual feedback
Solution Approach 1:
The patent separates the keyboard display functionality from the VNC screen capture layer by implementing the keyboard overlay in a different dimensional layer of the user interface hierarchy. The keyboard layout is rendered in an upper UI layer that is semi-transparent, while the VNC screen content remains in the base layer. This dimensional separation allows the keyboard guidance to be visible without compromising the quality or clarity of the screen recording, as the overlay does not interfere with the actual VNC screen capture process.
Data Source
AI summary
Methods, systems, and techniques are provided for displaying objects in virtual network computing (VNC). For example, a VNC connection may be established between a first device and a second device, where the VNC connection enables a synchronization of an interactive display layout from the first device to the second device. Subsequently, after the VNC connection is established, a page structure of the first device may be retrieved based on an application programming interface (API) on the second device. In some embodiments, based on the retrieved page structure, one or more non-interactive objects on the second device may be displayed, where the one or more non-interactive objects are displayed on top of at least a portion of the interactive display layout at the second device.