Application Remoting With Draw Commands for Low-Bandwidth Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing application remoting solutions, such as RDP and DOM mirroring, face inefficiencies in bandwidth usage, latency, and security vulnerabilities due to reliance on specific operating systems and susceptibility to malware attacks, especially in web browsers, which require extensive CPU, GPU, and memory resources.
Innovation Solution
The Adaptive Rendering Application Isolation System (ARAIS) transfers rendering data using draw commands instead of pixels, intercepting at a higher level in the graphics pipeline, allowing secure execution on a remote server and local rendering, minimizing bandwidth and latency, and supporting diverse operating systems without additional coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pixel-based remoting protocols (RDP) are used to forward desktop output, then compatibility with desktop operating systems is achieved, but bandwidth consumption increases and latency worsens
Solution Approach 1:
The patent extracts the rendering output from the pixel level and takes it out at the draw command level, where graphical operations are still in vector form. This allows transmitting only the essential drawing instructions rather than complete pixel data, significantly reducing bandwidth consumption while maintaining rendering quality.
Solution Approach 2:
The patent introduces an intermediary layer that intercepts draw commands between the rendering engine and the display output. This mediator captures graphical operations at the command level, translates them into a transmittable format, and forwards them to the client for local execution, avoiding direct pixel transmission.
2Reliability
If web browsers are executed remotely on a server to provide security isolation, then security vulnerabilities are reduced, but extensive CPU, GPU, and memory resources are required
Solution Approach 1:
The patent segments the browser execution environment into a remote isolated instance on the server that handles only draw command generation, while the resource-intensive rendering and display operations are performed locally on the client device. This segmentation reduces the resource burden on the server while maintaining security isolation.
Solution Approach 2:
The patent substitutes the traditional pixel-based graphical transmission mechanism with a draw command-based system. Instead of transmitting heavy pixel data that requires extensive processing and bandwidth, lightweight vector draw commands are transmitted and executed locally, dramatically reducing computational and memory resources required on the server.
3Ease of operation
If RDP protocols are used for application remoting, then desktop interface access is enabled, but security vulnerabilities to operating system interfaces are exposed
Solution Approach 1:
The patent introduces a draw command interception layer that acts as an intermediary between the remote browser instance and the operating system graphical interface. This intermediary captures graphical operations at the draw command level, preventing direct access to the OS interface and its associated security vulnerabilities, while still enabling desktop-like display output.
Solution Approach 2:
The patent extracts the graphical output generation from the operating system interface layer and takes it out to the application level where draw commands are intercepted. This extraction removes the security exposure to OS interface vulnerabilities while preserving the ability to display graphical content.
4Loss of energy
If draw commands are used instead of pixels for rendering data transfer, then bandwidth consumption is reduced, but compatibility with non-Windows operating systems becomes challenging
Solution Approach 1:
The patent implements a universal draw command interception mechanism that works across different operating systems and browser engines. The system captures graphical operations through standardized interfaces and translates them into a platform-independent format, enabling the same draw command transmission approach to function on Windows, Linux, macOS, and mobile operating systems without requiring OS-specific implementations.
Data Source
AI summary
A server receives from a client device that is executing a client application a request to initiate a remote application in the server. The server instantiates an instance of the remote application. The server intercepts draw commands associated with the remote application instance. The server provides the draw commands to the client to cause the client application to render portion(s) of output based on the draw commands. The server receives an input event from the client application. The server provides the client one or more draw commands based on the input event to cause the client application to render portion(s) of output based on those draw commands.