Batch Optimized Rendering and Fetch Architecture

Abstract

Implementations include a batch-optimized rendering and fetching architecture. An example method performed by the architecture includes receiving a request from a batch process to render a web page and initializing a virtual clock and a task list for rendering the web page. The virtual clock remains stationary while the request for the embedded item is outstanding and when the task is ready to run. The method may further include: generating a rendering result of the webpage when the virtual clock matches the running time of the stopped task in the task list, and providing the rendering result to the batch process. Another example method includes receiving a request from a batch process to render a web page, identifying an embedded item in the web page, and determining, based on a rewrite rule, that the embedded item has content that duplicates content of a previously retrieved embedded item.

Description

Background technique [0001] The World Wide Web is a rich source of information. Today, there are estimated to be over a trillion unique web pages. Many of these pages are dynamically created (eg, the home page of the New York Times) and have links to embedded content such as images and videos, which can affect the content and appearance of the rendered web page. For example, when a browser executes scripts such as JavaScript code, this can affect how a web page looks to a user and change the content and / or visual appearance of the page after the browser has finished rendering the web page. As another example, some web pages use style sheets, which tell the browser how to change the appearance of text. A typical web page may have hundreds of such additional embedded items, some of which are specifically designed for or targeted to the browser rendering engine. The additional information generated by the rendering process can be helpful to downstream systems such as internet ...

AI Technical Summary

Problems solved by technology

Although it is relatively simple for a single user's web browser to render a single web page in real time, rendering a large number of pages in real time (such as all pages on the World Wide Web (1 trillion pages) or even just the top 1% of the World Wide Web pages (10 billion pages)) would be much harder

Images