Server-Side Virtual Screen Change Capture for Web Application Publication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The continuous streaming of video data for rendering user interfaces in web applications consumes significant network bandwidth and computing resources, leading to inefficiencies in both client and server sides.
Innovation Solution
Implementing a server-side agent to capture and compare virtual screen frames, generating images of changes only, which are sent to the client device for re-rendering, reducing the need for continuous video streaming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If continuous video streaming is used to render user interface on client device, then the user interface rendering is smooth and continuous, but network bandwidth consumption and computing resources increase significantly
Solution Approach 1:
The patent extracts only the changed portions from the video frame and transmits them to the client device, rather than transmitting the entire frame. This is achieved by capturing the video frame, comparing it with the previous frame, identifying changed regions, and generating images of only these changed regions for transmission to the client device, thereby reducing network bandwidth consumption while maintaining rendering continuity.
Solution Approach 2:
The patent segments the video frame into changed and unchanged regions by comparing the current frame with the previous frame. Only the changed segments are captured, processed, and transmitted to the client device, while the unchanged segments are retained from the previous frame. This segmentation approach reduces the amount of data transmitted over the network while preserving the complete visual information needed for continuous rendering.
2Reliability
If continuous video streaming is used to render user interface on client device, then the user interface rendering is smooth and continuous, but computing resources on both client and server sides are consumed heavily
Solution Approach 1:
The server-side agent extracts only the changed portions from the video frame using frame comparison and change detection algorithms, rather than processing and transmitting entire frames. This reduces the computing workload on both the server side (for video processing) and the client side (for rendering), while still providing continuous visual updates for smooth user interface rendering.
3Loss of energy
If only changed regions are sent to client device, then bandwidth consumption is reduced, but the system complexity increases due to frame comparison and change detection
Solution Approach 1:
The system performs preliminary frame comparison and change detection on the server side before transmission. By pre-identifying and extracting changed regions server-side, the client device receives already-processed change images that require minimal additional processing. This preliminary action shifts computational complexity to the server, which is better equipped to handle it, while keeping the client device simpler.
Data Source
AI summary
Provided are computer-implemented methods and systems for optimization of publication of an application to a web browser. An example method for optimization of publication of an application to a web browser may include capturing, by a server-side agent, a video frame of a virtual screen associated with the application. The method may further include comparing, by the server-side agent, the video frame to a previous video frame of the virtual screen to detect a change in the video frame. The method may continue with generating, by the server-side agent, an image of the change based on the detection. The method may further include sending, by the server-side agent, the image of the change to a client device. The virtual screen may be re-rendered on the client device based on the image of the change and the previous video frame.