Time-dependent machine-generated cues

Abstract

Embodiments seek to improve pre-fetch hints using time-dependent, machine-generated hints. Some embodiments operate in the context of a hinting machine communicating with a client machine, which can develop information about whether and how to use resources in network transactions over time by collecting "resource samples." Each resource sample can identify resource presence state information at a sampling time. Time-based samples can be used to compute time-based probabilities for resources, which, for example, indicate the likelihood of a resource being used to render a web page at some subsequent time. Time-dependent hints can be generated in dependence on the time-based probabilities, and the time-dependent hints can be used to improve pre-fetching by optimizing hint information for a particular request time (e.g., pre-fetch hints for rendering a web page can be generated in a way that causes the web page to be rendered).

Description

TECHNICAL FIELD

[0001] Embodiments relate generally to network communication performance, and more particularly to time-dependent, machine-driven hint generation. BACKGROUND

[0002] User selection of web content and receipt of the web transmission of objects is a core part of the Internet experience for Internet users. While the user's experience is typically a single selection followed by viewing the web page rendered on the screen, the process of rendering the web page on the screen can involve a large number of resources (e.g., page objects) and multiple request / response roundtrip communications from the user's system to one or more network servers providing the web page resources. Additionally, each resource can be associated with a number of different stages as part of including the resource (or objects associated with the resource) in the web page rendered to the user. Each resource that is part of the web page and each stage associated with each resource can contribute to the overall page load time experienced by the device user as a delay. Various techniques allow for sending information to a browser about resources used to render a web page ("hints") and the browser can use these hints to improve the load time of that web page. SUMMARY

[0003] Among other things, systems and methods are described herein for using time-dependent, machine-generated hints to improve pre-fetching hints. Some embodiments operate in the context of a client machine having a page fetcher (e.g., implemented in a web browser) in communication (e.g., over a communication network) with a content server and a hint machine. The hint machine can develop information about whether and how to use resources (e.g., to render a web page) in network transactions over time by collecting "resource samples." Samples can be collected as part of receiving a hint request from the client machine, as part of receiving hint feedback from the client machine, as information monitored and / or generated during automated crawling of web pages, and / or in any other suitable manner. Each resource sample can identify presentation state information for the resource at the time the sample was collected (i.e., the sampling time). For example, a resource sample can identify whether the resource was loaded as part of rendering a web page, when the resource was loaded into the context of other resources on the page, and / or any other information describing whether and how the resource was loaded in relation to rendering the web page at the sampling time.

[0004] In this context, a resource sample can be considered a "time-based sample" in cases where the resource sample corresponds to a particular sampling time. Time-based samples can be used to compute time-based probabilities for the resource (e.g., as an instantaneous probability at a particular time, as a probability function over time, etc.). For example, such probabilities can indicate the likelihood that the resource will be used to render a web page at some subsequent time, and can be used to generate time-dependent hints. In some implementations, time-dependent hints can be used to improve pre-fetching by optimizing hint information for a particular request time (i.e., generating hints in a manner that causes a web page to be rendered). In other implementations, time-dependent hints can allow a client machine to generate their own hints as needed, even without access to a hinting machine (and / or external hinting information is unavailable). For example, an embodiment can receive a hint request at a request time, and generate and transmit a time-dependent hint based on time-based probabilities and the request time. Other implementations and features related to time-dependent hints are described more fully herein. BRIEF DESCRIPTION OF DRAWINGS

[0005] The present disclosure is described in relation to the accompanying drawings:

[0006] Figure 1 An illustrative communication system environment providing context for various embodiments is shown;

[0007] Figure 2A A block diagram showing a portion of an illustrative communication environment for implementing time-dependent hint generation according to various embodiments is shown;

[0008] Figure 2B A block diagram showing a portion of another illustrative communication environment for implementing time-dependent hint generation according to various embodiments is shown;

[0009] Figure 3 An example graph showing a time-based probability function for an illustrative resource is shown;

[0010] Figures 4A to 4D A graph showing a set of resource samples that can be fit to a presence model of an illustrative type is shown;

[0011] Figure 5 An illustrative system for implementing a data stream according to various embodiments is shown;

[0012] Figure 6 A flow diagram showing an illustrative method for time-dependent machine generation of hints according to various embodiments is shown;

[0013] Figure 7 A flow diagram showing another method for time-dependent hints according to various embodiments is shown;

[0014] Figure 8A schematic diagram of one embodiment of a computer system that can perform the methods of the invention described herein, and / or that can be used as any part of a client machine, content server, hinting machine 130, or any other such computer or device, is provided; and

[0015] Figure 9 A schematic diagram of a network system that can be used in accordance with a set of embodiments is shown.

[0016] In the drawings, like reference numerals can be used to denote similar components and / or features throughout the several views. Additionally, various components of the same type can be distinguished from one another by a second number following the first reference number, e.g., 21 2 and 21 3. If only the first reference number is used in the specification, the description is applicable to any one of the similar components having the same first reference number, unless otherwise indicated. DETAILED DESCRIPTION

[0017] In the following description, numerous specific details are set forth to provide a thorough understanding of the present application. However, it is understood that the present application can be practiced without the specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to obscure the present application.

[0018] Embodiments operate in the context of machine-driven hint generation, which provides time-dependent hints based on accumulated resource samples collected over time from one or more "client" machines. As used herein, "client" or "client machine" is generally intended to include any computing platform involved in a network transaction (e.g., rendering a web page), and is not intended to be limited to any particular network architecture, protocol, etc. For example, in some implementations, a client machine can interact with a server machine (e.g., a hinting machine) in the context of a "client-server" (e.g., a client-server architecture using a client-server type of protocol). Other implementations can operate in a peer-to-peer context or any other suitable context. In such cases, a particular machine can act as a client for a particular transaction, and as a server or other function in other transactions (e.g., the same machine can act as a "client" for one transaction and a server for another transaction, according to peer-to-peer and / or other protocols).

[0019] As used herein, a "web transaction" generally refers to a communication between a client machine and another machine (e.g., a server computer) to transfer a plurality of objects to the client machine, which can be presented to a user as part of a web page. As used herein, a "web page" is intended to broadly refer to any type of page sent over a communication network and composed of a plurality of page resources. For example, a web page can be a typical web page used in World Wide Web communications, a page (e.g., screen) of an application (e.g., application program, etc.), or any other type of web page. Moreover, a reference to "the web" is not limited to the Internet or World Wide Web; rather, "the web" can include any public or private communication network. Furthermore, terms used herein such as "page extractor," "page renderer," or "page loader" are not intended to be limited to any particular process in a web browser; rather, such terms can refer to any process or group of processes used to load and / or render a web page and its resources in a browser or other application (i.e., "render" and "load" are used herein to generally mean using resources to formulate a page). In one example, a web page can comprise a web browser page; a page extractor can comprise a web browser; and resources can comprise uniform resource locators (URLs), hypertext markup language (HTML) objects, scripts, cookies, and / or other server-side objects used by the web browser (e.g., required in some or all instances) to render the web page. In another example, a web page can comprise a screen of an application program (e.g., or any other application); a page extractor can comprise the application program (e.g., the portion of the application program that handles input / output interactions); and resources can be audiovisual content of the rendered screen.

[0020] Embodiments operate on "resources" and resource samples. As used herein, the term "resource" generally refers to a data element (e.g., file, script, etc.), a collection of data elements (e.g., web page, etc.), or an identifier of a data element or collection of data elements (e.g., uniform resource locator (URL), script call, etc.). In certain instances, a resource can be embedded in other resources and / or can comprise a collection of other resources. Embodiments of a "resource sample" generally indicate the corresponding resource (e.g., by reference to an identifier of the resource) and resource state information, but generally do not include the underlying data elements or elements themselves.

[0021] Embodiments are described in the context of hints, hint information, and the like. As used herein, a hint generally includes any information about resources provided to a page fetcher (or any suitable component of a client machine or a proxy system of a client machine) for rendering a web page to help improve the page fetcher's page load timing for that web page. The information can include: a list of all resources requested as part of a transaction, a list of resources needed to render an initially incomplete web page on the output of a client device, a set of cookies associated with the client device or processes operating on the client device (and / or hashed versions of those cookies), a set of cookies associated with one or more web page resources or client processes (and / or hashed versions of those cookies), a set of timings associated with each resource, a set of timings associated with the overall page rendering process, a set of relationships between resources, details associated with cached resources, resource sizes, resource types, resource fingerprints or checksums, resource locations on a page, cookie metadata, redirection chains, details of alternative content sources users (e.g., content delivery networks) that were available for some resources during a transaction, domains used during a transaction (including the number of objects expected to be fetched for each domain), security connection metadata, secure sockets layer (SSL) server certificate and / or revocation list information, and / or any other such details.

[0022] In various embodiments, after a page fetcher has completed rendering a web page and / or presenting the web page to a user, it can provide hint feedback information, which can include and / or be used to derive any hint information for subsequent web page transactions (e.g., including any of the hint information described above). The feedback information can be captured in any suitable manner, including by the client machine, by the page fetcher operating on the client device, by the web server, by a proxy server in the communication path between the client device and the web server, by an automated page fetcher under the control of a hint service, or by any other device involved in the web page transaction. The hints can be used to improve the web page load time in web page transactions. For example, the improvement can be realized by reducing the overall time from a user selection via the page fetcher to completion of rendering the web page by the user in response to the selection. Such an improvement can also be realized by reducing the initial time to render an incomplete version of the web page, which can be useful for user purposes. In one possible embodiment, the overall time reduction can result from using latency information in conjunction with other feedback information to determine how aggressively the page fetcher attempted to pre-fetch sub-resources as part of future web page transaction instances.

[0023] Examples of hint and feedback information can be found in U.S. Patent Application Serial No. 14 / 729,949, entitled "Server-Based Embedded Web Page Feedback and Performance Improvement," U.S. Patent Application Serial No. 13 / 372,347, entitled "Browser-Based Feedback for Optimized Web Browsing," U.S. Patent No. 9,037,638, entitled "Assisted Browsing Using Hinting Functionality," U.S. Patent Application Serial No. 14 / 212,538, entitled "Faster Web Browsing Using HTTP Over TCP Transport," U.S. Patent Application Serial No. 14 / 276,936, entitled "Cache Hinting System," and U.S. Patent Application Serial No. 14 / 729,949, entitled "Server Machine-Driven Hint Generation Using Client Machine-Driven Feedback to Improve Web Page Loading," each of which is expressly incorporated by reference herein for all purposes.

[0024] Resources used in such web pages can include HTML files, Cascading Style Sheet (CSS) files, image files, video files, or any other such files. References to different instances of a web page transaction refer to transactions performed by different client machines at different times, or the same transaction performed by a single client machine at different times. Different instances of these web page transactions can include variations in resources that are part of the web page transaction, which can be due to customization across different client machines, or updates to the web page over time. Further, different web pages and different web page transactions can include the same or similar resources. In certain embodiments, feedback information and hints generated for resources seen in one web page transaction can be applied as hints in transactions for separate web pages if the root URLs are similar, or if there is a sufficient degree of commonality between the resource groups for the two web page transactions. Similarly, as used herein, terms such as "rendering" and "loading" are used broadly (and, in most cases, interchangeably) to refer broadly to allowing a user to interact with page resources via a page fetcher interface. For example, rendering or loading can include displaying and / or formatting in the context of static visual content, playing in the context of video or audio content, executing in the context of code or other scripts, and so on.

[0025] Further, as used herein, a "root" refers to an initial portion of a web page transaction that is directly initiated by a user selection or action or that is anticipated to be initiated by a user action. For example, a user clicking on a web page link initiates a root request for that link. A root response is a response that is directly responsive to that root request. The root response also includes a root resource. The root resource includes information that enables a page scraper to directly or indirectly identify other resources needed to render and present a complete web page. In some cases, a "root" resource can include a primary sub-resource (e.g., sub-resource) in an inline frame (iframe) on a page, etc. (e.g., where each of multiple inline frames is separately prompted from different content servers).

[0026] A "redirect" refers to a response to a root request that directs the requesting client device to a single resource that has at least one sub-resource. For example, a client device can send a root request and receive back a redirect response. The client device can then send a redirect sub-request to the redirect target indicated in the redirect response. In certain embodiments, a response to the redirect sub-request can include a feedback script or prompt. Thus, although certain embodiments describe operations with root requests and responses, in various embodiments, any root, sub, or redirect response described herein can include a feedback script as described in various embodiments herein.

[0027] A "sub" request and response is a subsequent request and response that is directly or indirectly caused by an embedded or computed reference to a root resource or other resource in other sub-resources. Sub-resources, requests, and responses are always one or more steps removed from a user action by a root that is directly responsive to the user action. A sub-resource can include a reference to additional sub-resources, resulting in a series of requests and responses. Each of the above requests and responses can be hypertext transfer protocol (HTTP) requests and responses, including HTTP headers and related messages. In various embodiments, other communication protocols can be used.

[0028] Figure 1 An illustrative communication system environment 100 that provides context for various embodiments is shown. The communication system environment 100 includes a client machine 110, a content server 120 (e.g., web server), and a prompt machine 130 that communicate over a communication network 140. The network 140 can include any one or more suitable communication networks and / or communication links, including any wide area network (WAN), local area network (LAN), private network, public network (e.g., the Internet), wired network, wireless network, and so on. Typically, the communication system environment 100 can include many client machines 110 that connect with multiple content servers 120 over the communication network 140.

[0029] As described herein, content server 120 can communicate with one or more hint machines 130 directly and / or via communication network 140, and / or client machines 110 can communicate with hint machines 130 directly and / or via communication network 140 (e.g., at the direction of the content server), in accordance with various embodiments. Some embodiments are directed to improving the loading and rendering of resources that make up web pages, screens of applications, and / or other similar web page contexts. In such contexts, client machines 110 can typically make requests for web pages provided (e.g., hosted) by content server 120. Loading and rendering the requested web pages can involve subsequently requesting and receiving a plurality (sometimes a large plurality) of resources (e.g., visual content, audio content, executable scripts, etc.) that make up the web pages. The loading and rendering of such web pages can be improved by requesting the resources at particular times (e.g., by pre-fetching the resources in a particular order, etc.), and the client machines can be instructed to use "hints" to achieve such improvements, as described herein. The resources can be identified in hint groups by URL, a combination of URL and regular expressions, scripts, or other similar techniques. The loading and rendering of such web pages can also be improved by hints that support pre-resolving domain names, pre-establishing TCP connections, pre-establishing secure connections, pre-determining and minimizing redirection chains, and similar functionality that can be performed prior to content loading to improve overall page loading performance. Furthermore, the probability that a resource is needed and the priority that the browser should give it can be communicated to further improve page load times. Additionally, various image, video, and document formats associated with a given resource can be pre-sent to the device as hints, thereby allowing the renderer to dynamically adapt to network conditions and constraints and minimize data traffic associated with pre-fetched resources. Furthermore, hints can direct the selection of CDN, cache, or other server locations to improve page load times.

[0030] Client machines 110 can be implemented as any suitable computing device having storage resources, processing resources, and network communication resources. For example, client machines 110 can be desktop computers, tablet computers, laptop computers, mobile phones, personal data assistants, wearable devices that use a network, home appliances that use a network, etc. Each client machine 110 includes one or more page extractors 115. Page extractors 115 can include any system implemented in a client machine 110 that allows web page transactions and is used at least in part to render web pages and present them to a user via an output device of the client machine 110.

[0031] Content servers 120 can generally include any one or more computing environments for providing (e.g., hosting and / or otherwise providing access to) web page content to client machines 110 via communication network 140. For example, content servers 120 can include web servers, content distribution networks (CDNs), caches, and the like. As shown, content servers 120 can include one or more data storage systems storing web pages 125 or be in communication therewith. As described herein, it is assumed that web pages 125 are composed of multiple resources 127. For example, loading one of web pages 125 can involve requesting, receiving, and rendering resources 127 that make up web page 125. Some or all of resources 127 of web pages 125 provided by content servers 120 can be stored in data storage systems, or some or all of resources 127 can be stored remotely from content servers 120.

[0032] One or more hint machines 130 can be implemented as one or more standalone server computers, as part of one or more content servers 120, and / or in any other suitable manner for maintaining and updating hint information 135 (e.g., from hint feedback 147 from client machines 110, from hints 150 computed from hint information 135, etc.). Hint information 135 can be stored in one or more data stores that are part of, coupled or in communication with hint machines 130, or in any other suitable manner. Embodiments support different types of hint information 135 and hints 150 generated therefrom, including, for example, information about which page objects 127 are needed to render web pages 125, timing information about those page objects 127 (e.g., order and timing at which page objects 127 should be requested), and the like. Hint information 135 can be maintained, computed, updated, etc. in any suitable manner, including from hint feedback 147 received from one or more client machines 110. Embodiments of hint machines 130 apply machine learning techniques to hint feedback 147 from multiple related web page transactions (e.g., multiple instances from multiple client machines 110 rendering the same (or sufficiently similar) web pages). Received hint feedback 147 can be used to refine, perfect, update, strengthen, or otherwise improve machine-driven hint models maintained by hint machines 130, thereby facilitating the generation and communication of optimized hints.

[0033] The client machine 110 can present the requested web page 125 according to hints 150 generated from the hint information 135, which effectively predicts what resources 127 the client machine 110 will need when to optimally present the web page 125; the client machine 110 can actually present the web page 125 according to at least the received hints 150; the client machine 110 can monitor the actual presentation of the web page 125 to determine which resources 127 were actually used to present the page according to which timings; and the monitored information can be fed back (i.e., as hint feedback 147) to the hint machine 130 for updating the hint information 135 and refining the generation of future hints 150. The hints 150 can be generated for and / or applied to present particular web pages 125 from which the hints were derived, other web pages 125 from the same or similar domains, web pages 125 that differ only by a portion of their path (or dynamically generated fields, etc.), and / or the like. For example, hint information 135 collected from a particular web page 125 (e.g., from time-based samples) can suggest time-dependent trends and / or other information about all web pages in a particular domain (e.g., "breakingnews.com" pages have a banner at the top of the page that is the same on all pages in the domain and changes very infrequently, and they have a primary story image that is in the same location on each page in the domain with different content, or the like).

[0034] Some embodiments generate and process hints in a computationally generated manner (e.g., generated by a machine based on feedback and analysis, rather than manually generated by an encoder based on assumptions). Such machine-generated hints can be enhanced by increasing time dependence, as described herein. Embodiments of the hint machine 130 can develop information about whether and how to use resources (e.g., to present web pages) in network transactions over time by collecting samples of resources. Samples can be collected as part of receiving hint requests from client machines (e.g., as part of page requests 141), as part of receiving hint feedback 147 from client machines, as information monitored and / or generated during automated crawling of web pages (e.g., by the hint machine 130 itself), and / or in other suitable manners. Time-based samples can be used to compute time-based probabilities of resources, which can be used to generate time-dependent hints.

[0035] Figure 2A A block diagram showing a portion of an illustrative communication environment 200a for implementing time-dependent hint generation according to various embodiments is shown. FIG. 2 shows a client machine 110 in communication with a hint machine 130 over a network 140, which can be the network 140 described above with reference to FIG. 1. The client machine 110 can be a client machine 110 described above with reference to FIG. 1, and the hint machine 130 can be a hint machine 130 described above with reference to FIG. 1. The client machine 110 can be configured to present a web page 125 according to hints 150 generated by the hint machine 130, and the hint machine 130 can be configured to generate the hints 150 based on information collected from the client machine 110 and / or other sources. Figure 1Embodiments of the described system. Some of the description relates to communication between components of the client machine 110 and components of the hint machine 130, however these are merely as general illustrations of functionality and connections. Reference is made to Figure 1 As described and generally shown in FIG. 2, the hint machine 130 can communicate directly with the client machine 110 (through the network 140), only via one or more content servers 120 with the client machine 110 (e.g., where the hint machine 130 communicates with the content server 120 through one or more networks 140 and / or is part of the one or more content servers 120), with one or more content servers 120 and the client machine 110 through one or more networks 140, etc. For example, hint functionality can be processed between the client machine 110 and the hint machine 130 without involving any content servers 120, only through one or more content servers 120, or in any suitable combination.

[0036] As shown, the client machine 110 can include a page fetcher 115, such as a web browser. Embodiments of the page fetcher 115 can include a presentation engine 210, a resource engine 220, and a client hint subsystem 230. The presentation engine 210 can present resources of a web page for consumption (e.g., display, etc.) via a graphical user interface (GUI) 215 of the client machine 110. For example, the presentation engine 210 can process HTML code, scripts, page objects, etc. to effectively provide a user experience of a web page via the GUI 215.

[0037] When a web page is requested, the resource engine 220 can generate requests for resources of the requested web page, transmit the requests to one or more content servers 120 through the network 140, receive resources in response to the requests, and process the responses. For illustrative purposes, a user can request a web page via the GUI 215 (e.g., by entering a web address), the resource engine 220 can obtain some or all of the resources required to present the requested web page (e.g., according to HTML code, scripts, cookies, page objects, etc.), and the presentation engine 210 can process the obtained resources to effectively provide a user experience of the requested web page via the GUI 215 (present the web page using the resources).

[0038] As described herein, embodiments of the page extractor 115 can use the client hinting subsystem 230 to utilize hints. The hinting functionality can be utilized at any or all stages in a web transaction. One stage is the web page request stage, during which various resource requests can be made to one or more content servers 120 (e.g., by the resource engine 220), and requests for hints related to those resources can be made to the client hinting subsystem 230 (e.g., by the client hinting subsystem 230) comparable to those resource requests. For example, in response to a user requesting a web page (or in response to predicting that a user will request a web page), the resource engine 220 can begin requesting resources (e.g., resources related to root URLs and sub-URLs), and the client hinting subsystem 230 can make one or more requests to the hint machine 130 indicating the URLs of those resources, seeking related hints. Another stage is the feedback stage. As resources are being loaded for a web page, as the page is being rendered, etc., the client hinting subsystem 230 can collect feedback information, as described above (e.g., about which resources are involved in rendering the web page, about timing information for resources, etc.). After the web page has been rendered by the rendering engine 210 (or during rendering, after being presented to the user via the GUI 215, after multiple pages have been rendered and aggregated feedback has been collected, or at any other appropriate time), the client hinting subsystem 230 can send hint feedback to the hint machine 130 for use in generating future hints for the web page and / or resources (e.g., any web pages that call those resources).

[0039] Machine-generated hints by the hint machine 130 can generally involve aggregating and maintaining information about how and when resources are used to render web pages. For example, a hint service can use so-called "waterfall data" from multiple past visits to a web page to compute a probability that a given resource (e.g., a sub-URL of a web page) will be used in a future visit. One technique to compute such a probability is to divide the number of occurrences of a given resource by the number of visits to the web page on which the resource is used, thereby effectively yielding a likelihood that the resource will be used subsequently to render the web page. For example, under the assumption that more recently sampled pages are less likely to change, the probability can be improved by more strongly weighting the presence or absence of a resource in more recent visits (e.g., even assuming that a web page changes over time, it is generally believed that such changes are less likely to occur after a shorter period of time has passed). However, such a probability does not record or lead to actual time dependencies of resources on a page. For example, some resources can change more or less frequently than others, and such changes can exhibit some periodicity or other time characteristics, can be indicative of time characteristics of other related resources, etc. Embodiments described herein accumulate time-based information about resource rendering, from which time-based hint probabilities and time-dependent hints are developed and utilized. Such time-dependent hints can provide many features, as described herein.

[0040] Embodiments of the hint machine 130 include a server hinting subsystem 270, which can include a resource sampler 240 and a time-dependent hint (TDH) generator 260. The resource sampler 240 can receive resource samples and can store the resource samples in a sample data store 280. In some embodiments, the resource sampler 240 can receive resource samples by monitoring network transactions involving the client machine 110 (e.g., and / or involving machines acting on behalf of the client machine 110, such as proxies). For example, when the client machine 110 issues a hint request, provides hint feedback, accesses a web page, etc., the resource sampler 240 can monitor which resources are used to render the web page and timing information related to loading those resources. In other embodiments, the server machine 130 (e.g., the resource sampler 240) can perform automated resource sampling. For example, the resource sampler 240 can include one or more "bots" or the like that can autonomously crawl web pages and collect resource samples, and / or direct their operation. Such an automated approach can include any suitable functionality for collecting resource samples, such as a full browser instance, a partial browser instance, etc. In some implementations, the automated browser approach can model the functionality of multiple types of browsers and / or multiple types of rendering platforms (e.g., for different operating systems, mobile vs. desktop environments, etc.).

[0041] Each resource sample is recorded at a respective sampling time and indicates a rendering state of a resource relative to rendering the web page at the respective sampling time (and can be received at the sampling time or some subsequent time). The rendering state can include any information about rendering the resource in the context of one or more network transactions (e.g., on one or more web pages). For example, when a user first accesses a news-related web page, a number of resources can be loaded, including text, images, videos, scripts, etc. Some resources relate to relatively static content of the web page (e.g., data elements that contribute to the overall look and feel of the page), and other resources are relatively dynamic content of the web page (e.g., data elements that relate to specific news stories, ad banners, etc.). Accordingly, when the user accesses the same page a second time, some resources (e.g., the more dynamic resources on the page) can have changed since the first time. Thus, the rendering state can indicate which resources were loaded at which sampling time.

[0042] Some embodiments of resource sampler 240 can collect additional presentation state information about resources as part of sample collection. Some implementations associate each sampled resource with one or more super-resources and / or sub-resources. For example, a group of resources can be sub-URLs of a particular logical context (e.g., a web page, a resource domain, a root domain, etc.), and implementations can store these resources in association with their logical context; or a particular resource can be a script or page that invokes one or more sub-resources, and implementations can store the resource using such an indication. Other implementations can store information in association with resource samples, such as file types, one or more resource indicators (e.g., URLs, one or more URLs identified as pointing to equivalent resources, etc.), resource presentation characteristics (e.g., size relative to a web page, location on a web page, interactivity with user devices and / or other resources on a web page, etc.). For example, a web page can be arranged with the top 10% of the page having content that never changes, the left 10% of the page having content that changes on the first day of each month, and the remaining page having highly dynamic content.

[0043] TDH generator 260 can compute, from stored resource samples, a time-based probability that each resource is extracted as part of a rendered web page over time. As resource samples are collected over time, some or all of the samples (e.g., resource presentation states at sampling times) can be added to a statistical model. The statistical model effectively computes a probability that a particular resource will be presented in association with a particular network transaction at a particular time (or times) in dependence on whether (e.g., and how, when, etc.) the resource was presented in association with other network transactions at previous sampling times. Figure 3 and 4A Figures 4A through 4D show some examples of time-based resource probability information for various embodiments.

[0044] Figure 3 An example graph 300 showing a time-based probability function 350 (i.e., probability 320 versus time 310) for an illustrative resource is shown. It is assumed that a number of samples have been collected over previous time frames and stored as "past data" 325. From past data 325, a time-based probability function 350 can be computed and plotted. For example, a statistical trend curve can be computed using past data 325. As shown, past data 325 can include some or all samples for a particular resource (and in some cases, some related resources as well) up to a "last sampling time" 330k, which can be a most recent time at which a sample was collected for the resource of interest.

[0045] The "time" 310 associated with the sample(s) and / or with the computation of the time-based probability function 350 can be expressed in any suitable manner. In some embodiments, the time 310 can be expressed as a relative time with reference to the request time 340. For example, past data 325 can be expressed as an amount of time in the past relative to the request time 340 and / or future predicted probabilities can be expressed with reference to an amount of time from "t=0", the request time, and the like. In other embodiments, the time 310 can be expressed with reference to an absolute time scale, such as a timestamp (e.g., a date and time, rounded to the nearest second, and the like). The absolute time can be based on the location of the hinting machine 130 (e.g., time zone), a pre-agreed time reference (e.g., Greenwich Mean Time, time measured from a defined epoch, and the like), and the like.

[0046] In some implementations, one or more additional mathematical considerations can be included in the computation of the time-based probability function 350. As one example, some implementations can apply a decay function to the time-based probability function 350 such that the predicted future probability decreases over time in a particular manner (e.g., by a predetermined slope, decay rate, half-life, and the like). In certain such implementations, the decay function can be applied to the instantaneous probability and / or any time-based probability computation (e.g., after the last data sample is collected, the probability for the resource can monotonically decrease according to the hint decay function). In certain cases, the application of the decay function does not require that the overall probability function be monotonically decreasing; rather, the hint decay function can produce a contribution that effectively decreases the time-based probability function 350 over time. One example of a hint decay function is a square function that starts at one (e.g., 100%) and decreases to zero (e.g., 0%) at some time, thereby representing an expiration time for a particular hint and the like. As another example of an additional mathematical consideration that can be included in the computation of the time-based probability function 350, some implementations can apply a weighting function to the resource samples in the past data 325 to make some samples more influential than others. The weighting can be based on "freshness" (e.g., more recent samples can be more heavily weighted than older samples), based on domain relationships (e.g., samples for a resource recorded in association with a requested domain can be more heavily weighted than samples for the same resource recorded in association with other domains), based on platform relationships (e.g., samples for a resource recorded in association with a platform similar to the current requesting platform can be more heavily weighted than samples recorded for other platforms), and the like.

[0047] The time-based probability function 350 can be computed in any suitable manner and at any suitable time. For example, the time-based probability function 350 is recomputed each time a new sample is collected for a resource, periodically, in response to a prompt request, etc. As described herein, the computed time-based probabilities can ultimately be used to develop time-dependent prompts. Thus, in some cases, there is a "request time" 340 at which a particular domain, resource, etc. is requested, and the time-based probability function 350 is utilized. It can generally be assumed that the request time 340 is after the last sampling time 330k, such that the time-based probability function 350 is available to predict the time-dependent presence probability of the particular resource substantially at the request time 340 (i.e., an instantaneous probability) and / or at one or more future times (e.g., one or more instantaneous probabilities associated with particular future times, one or more average probabilities associated with one or more future time frames, one or more probability functions associated with one or more future time frames, etc.).

[0048] In some cases, the presence state of a resource can be modeled according to one or more presence models. Each presence model can effectively be a score, a parametric function, a machine learning model, a statistical model, etc. Some embodiments can attempt to statistically fit resource presence state data into one or more predetermined presence models. Figures 4A to 4D A plot 400 of a set of resource samples that can be fit into an illustrative type of presence model is shown. Each plot 400 shows samples of resource presence 420 of a particular resource at a plurality of sampling times 330 (e.g., from some sampling time 330a to some last sampling time 330k). For simplicity, each presence 420 is plotted in a binary fashion (e.g., as "present" or "not present"), and a line is drawn through the sample points. The plots 400 are not intended to limit the type of data that can be collected, the amount of data, etc., or the manner in which it can be expressed. For example, the sampling times 330 can not be equally spaced, there can be more than two dimensions (e.g., presence can result in other information, such as location, platform, etc., as described above), etc.

[0049] Turning to the first example, Figure 4ASamples of a resource under a "regular presence for short" type of presence model are shown. For example, the most recent N samples of this resource indicate presence, but some previous samples suggest absence. This can indicate that the resource has consistently been presented as part of a particular web page, but the consistency is only apparent for a short time. Some implementations can compute a time-based probability function 350 from such data to reflect a high probability that the resource is being presented with the web page, but the computed function can also indicate a lower probability and / or a more sharp decay in accordance with the short nature of the consistency. For example, the computed time-based probability function 350 can indicate a higher probability over time and / or less decay as the data remains consistent for a longer time. Further, presence models can be implemented to consider indications of occasional absence in various ways. For example, one implementation can consider a resource to no longer be "regular" upon a single sample of absence. Another implementation can consider a resource to be "regular" whenever there is a minimum threshold number of consecutive samples of presence. Another implementation can consider a resource to be "regular" whenever there are less than a maximum threshold number (or frequency) of consecutive samples of absence.

[0050] Turning to a second example, Figure 4B Samples of a resource under a "dropped" type of presence model are shown. For example, the most recent N samples of this resource indicate absence, while previous samples indicate presence. This can indicate that the resource was previously presented as part of a particular web page, but is apparently no longer part of the web page. Some implementations can compute a time-based probability function 350 from such data to reflect a low (or zero) probability that the resource is being presented with the web page. In certain implementations, the low probability can be further influenced (e.g., weighted) by the persistence of the absence indications (e.g., number of samples, freshness of samples, etc.). For example, the computed time-based probability function 350 can indicate a lower probability as the absence data remains consistent for a longer time.

[0051] Turning to a third example, Figure 4C Samples of a resource under a "one-time" type of presence model are shown. For example, the sample data shows a single (or limited number) of samples indicating presence (before and after) a large number of samples indicating absence. This can indicate that the resource has been presented once (or a limited number of times) as part of a particular web page, but is apparently not currently part of the web page. Some implementations can compute a time-based probability function 350 from such data to reflect a low (or zero) probability that the resource is being presented with the web page. In certain implementations, the low probability can be further influenced (e.g., weighted) by the persistence of the subsequent presence and / or absence indications (e.g., number of samples, freshness of samples, etc.).

[0052] Turning to a fourth example, Figure 4DResource samples are shown under a "ephemeral" or "periodic" type of presence model. For example, sample data shows a pattern of presence and absence over time. This can indicate that the resource tends to be presented as part of a particular web page at certain times (e.g., certain times of day, month, year, etc.) but not otherwise, but with a statistical confidence in a deterministic manner. Some implementations can compute a time-based probability function 350 from such data to reflect probabilities that substantially follow a deterministic pattern. For example, the computed time-based probability function 350 can follow a periodic function (e.g., a step function or a sinusoidal function, etc.) or the like. For the sake of illustration, such a periodic probability can exist in a web page that is updated with new content at a deterministic time (e.g., every Wednesday at 5:00 AM).

[0053] Figures 3 to 4D The examples provided are merely intended to illustrate certain types of persistence models and time-based probability functions 350, and are not intended to include all types or to limit the implementations to any number or type of models or functions. Moreover, embodiments can adjust the models and / or time-based probability function 350 computation in any suitable manner. As one example, the time-based probability function 350 can be computed from resource samples using a sliding window that defines a time frame of relevance, and the size of the window can be adjusted based on particular factors for different types of models or the like. As another example, persistence models that can or can not follow regular patterns can be developed using machine learning and / or similar techniques. For example, principal component analysis or other types of data analysis can be used to extract the most relevant and relevant way of persistence features; and those extracted features can be used to formulate persistence models (e.g., dynamically, periodically, etc.).

[0054] Returning to Figure 2ATDH generator 260 can use the computed time-based probabilities to generate time- dependent hints for pre-fetching resource groups in association with rendering a web page at a client machine. For example, after computing predictions of the existence probabilities of resources over time, these predictions can be utilized to determine which resources to pre-fetch (e.g., and / or in what priority, order, etc.) in response to a request to load a web page. The time-dependent hints for a particular resource can be generated in accordance with a respective time-based probability function 350 associated with that resource and the request time (e.g., which can be expressed as a set of data points, a mathematical expression, and / or any suitable manner). In some embodiments, the time-dependent nature of the hints can increase the likelihood that the most relevant resources will be fetched in an optimal order, thereby improving page load times and / or otherwise improving the performance of web transactions. In other embodiments, the availability of predicted time-based probabilities (e.g., functions, data points, etc.) can be utilized to generate time-dependent hints even when further information is not available at the request time, e.g., when the hinting machine 130 is inaccessible to the client machine 110, the hinting information is outdated, etc.

[0055] Embodiments of the hinting machine 130 can include a communication subsystem 250 that can transmit the generated time-dependent hints to one or more client machines 110. In some embodiments, the communication subsystem 250 can also receive and process communications related to hint requests and / or other hinting-related communications from the client machines 110, content servers 120, etc. For example, the communication subsystem 250 can receive a hint request from a client machine 110 in association with rendering a web page at the client machine 110 at a request time via the communication network 140; the TDH generator 260 can generate a hint response in response to the hint request, the hint response including time-dependent hints generated from respective time-based probabilities (e.g., computed prior to the request time and / or at the request time). The communication subsystem 250 can then transmit the hint response to the client machine 110 in response to the hint request.

[0056] In some embodiments, the TDH generator 260 can generate time-dependent hints in which an instantaneous hint probability for the one or more time-dependent hints is computed in accordance with the request time and respective time-based probabilities associated with the resources of the one or more time-dependent hints. For example, the instantaneous probability can be a computed probability at the request time in accordance with the time-based probability function associated with the resource. The communication subsystem 250 can then transmit the hint response by transmitting the time-dependent hints along with their associated instantaneous probabilities.

[0057] In other embodiments, the TDH generator 260 can generate time-dependent hints, where the one or more hint probability functions are computed by way of respective time-based probabilities of one or more resources associated with the one or more time-dependent hints. As described above (e.g., with reference to Figures 3 to 4D ), the hint probability functions can indicate probabilities that a resource will be requested by a requesting client machine at a time after the request time as part of rendering a web page. The communication subsystem 250 can then transmit the hint response by transmitting the time-dependent hints and the one or more hint probability functions. As shown, in some embodiments, further hint information can be stored in the same or another hint information store 135. The additional hint information can include any information useful for performing static, time-dependent, and / or other hint functions.

[0058] For further clarity, Figure 5 An illustrative system 500 for implementing data flows in accordance with various embodiments is shown. The system can include a means for receiving a plurality of resource samples 515. As described herein, the means can include any suitable hardware (e.g., and supporting software) for communicating over a network and / or loading resources to perform (or simulate, etc.) a network transaction, such as an antenna, physical and / or logical ports, protocols, modems, routers, etc. As shown, the means can include the resource sampler 240 and the resource sample store 280. The resource sampler 240 can collect samples 515 of resources in association with the bot-driven requests 503 and / or the client-driven requests 505 for network transactions (e.g., for loading web pages). The collected samples 515 can be stored in the resource sample store 280 in any suitable manner. For example, the samples 515 can be grouped by associated resources 510, etc. The sample data store 280 can include any suitable tangible storage hardware and supporting hardware and / or software.

[0059] Some embodiments of the resource sampler 240 can also include a resource analysis engine 520 for generating, fitting, and / or otherwise determining and utilizing resource relationships 525. For example, as described above, the resource relationships can include locations of resources on a page, groups of resources associated with the same page, resources of particular file types, etc. In some embodiments, the time-based probabilities and / or time-dependent hints for a particular resource can be determined by way of time-based probabilities of one or more other related resources and / or time-dependent hints. Some embodiments of the resource analysis engine 520 can also generate and / or fit presence models and / or other types of curves.

[0060] The system 500 further includes means for computing, for each resource 510, a time-based probability 535 that the resource 510 is fetched over time as part of rendering a web page from the resource sample 510; for example, the resource sample 515 and / or the resource relationships 525 can be provided to the means for computing, which can include a TDH generator 260 for generating time-dependent hints 550. For example, the means for computing can be implemented as a time-based probability engine 530 that is part of the TDH generator 260. Accordingly, the time-based probability engine 530 can include hardware (e.g., a hardware computing platform and associated software) for receiving the resource sample 515 and / or the resource relationships 525 and computing the time-based probabilities 535 for the sampled resources 510. The TDH generator 260 can further include means for generating time-dependent hints 550 from the respective time-based probabilities 535, the time-dependent hints for pre-fetching the set of resources 510 in association with rendering a web page at a client machine (e.g., the client machine 110). For example, the means for generating time-dependent hints 550 can include computing hardware and associated software for receiving the computed time-based probabilities 535 (e.g., along with additional hint information 135) and formulating the time-dependent hints 550 according to one or more data formats, protocols, etc.

[0061] In some embodiments, the client machine 110 can store the received time-based probabilities for client machine-driven hint generation. For example, in the event that the client machine 110 determines to prefer self-generated hints (e.g., in the event that it is determined that there is no connection to the hinting machine 130, where connection to the hinting machine 130 is determined to be slow or unreliable, etc.), the client machine 110 can use the stored time-based probability functions to generate its own hints for associated resources in a manner that results in time-dependence in the requested context. Embodiments of the client machine 110 can receive time-dependent hints from the hinting machine 130 over the communication network 140. The client hinting subsystem 230 (e.g., which can include a page fetcher and / or any other suitable components) can pre-fetch at least some resources in association with loading a web page and in accordance with the received time-dependent hints. In some embodiments, the client hinting subsystem 230 includes a hint store that can store the received time-dependent hints with their associated hint probabilities (e.g., the associated time-based probability functions) as computed by the hinting machine 130. In such embodiments, the client hinting subsystem 130 (e.g., the page fetcher) can pre-fetch some or all of the resources where a client-side hint set for those associated resources is computed from the request time and the stored hint probability functions, and the associated resources are pre-fetched in accordance with the client-side hint set.

[0062] Figure 2BA block diagram illustrating a portion of another illustrative communication environment 200b for implementing time-dependent hint generation in accordance with various embodiments is shown. Embodiments of the communication environment 200a can be implemented in a manner substantially similar or identical to that described with reference to Figure 2A the communication environment 200a described. Except that the server machine 130 is implemented as a portion of the client machine 110 in Figure 2B Figure 2A 2B and are identified by the same reference numerals. Such labeling is intended to represent functional similarity and not limitation of possible implementations of the components.

[0063] For example, the resource sampler 240 and the TDH generator 260 can be implemented as a portion of the hint processor 270. As described for the resource sampler 240 and the TDH generator 260, the hint processor can include any hardware and / or software for implementing time-dependent hints. Embodiments of the client machine 110 with the integrated hint machine 130 can be used to provide various features. One such feature is to provide a peer-to-peer hint service. For example, the hint machine 130 integrated in the client computer 110 can be used for a point-to-point aggregation of hints across some or all of the peers, for distributed machine-driven hint generation, etc. Another such feature is for the client to generate hints based on its own hint information. For example, the client machine 110 can maintain its own hint feedback and / or collect hint feedback from other client machines 110 (e.g., periodically through some synchronization process, using a dictionary or client model, etc.), and the client machine 110 can generate its own hints as needed without consulting any other client machine 110 or hint machine 130.

[0064] Figure 6 A flow diagram of an illustrative method 600 for time-dependent machine-generated hints in accordance with various embodiments is shown. Embodiments of the method 600 can be implemented by any suitable system, such as the system described with reference to Figures 1 to 3 The method 600 can begin by receiving resource samples (e.g., by a hint machine) at stage 604. Each resource sample can be recorded at a respective sampling time and can indicate a rendering state of a resource relative to rendering a web page at the respective sampling time. At stage 608, embodiments can compute a time-based probability that each resource is fetched over time as part of rendering the web page from the resource samples. At stage 612, embodiments can generate a time-dependent hint from the respective time-based probabilities, the time-dependent hint for pre-fetching a group of resources in association with rendering the web page by a client machine.

[0065] Figure 7 ​A flow diagram illustrating another method 700 for time-dependent hints in accordance with various embodiments is shown. Embodiments of the method 700 can begin at stage 704 with receiving a hint request (from a client machine by a hint machine) in association with a request time and rendering a web page by the client machine. At stage 708, embodiments can generate a hint response including time-dependent hints (e.g., at least some of the time-dependent hints generated in stage 616 of the method 600) in response to the hint request. At stage 712, embodiments can transmit the hint response (e.g., from the hint machine) to the client machine in response to the request. According to some embodiments of the method 700, the generating of stage 708 can include computing a momentary hint probability for the one or more time-dependent hints as a function of the request time and respective time-based probabilities of resources associated with the one or more time-dependent hints (shown as 708a). In such embodiments, the transmitting of stage 712 can include transmitting the time-dependent hints and the momentary probability (shown as 712a). According to other embodiments, the generating of stage 708 can include computing a momentary hint probability function as a function of respective time-based probabilities of resources associated with the one or more hints such that the hint probability function indicates probabilities of the resources being requested by the requesting client machine as part of rendering the web page at a plurality of times after the request time (shown as 708b). In such embodiments, the transmitting of stage 712 can include transmitting the time-dependent hints and the hint probability function (shown as 712b). Figure 6

[0066] Figure 8 A diagrammatic representation of one embodiment of a computer system 800 is provided that can execute the methods of the application described herein, and / or can be used as any portion of a client machine 110, content server 120, hint machine 130, or any other such computer or device. It should be noted that, Figure 8 only intended to provide a general description of the various components that can be present in any or all of the components. Thus, Figure 8 the individual system elements are broadly described as being implemented in relatively separate or relatively more integrated manners.

[0067] The computer system 800 is shown comprising hardware elements that can be electrically coupled via a bus 805 (or can otherwise be in communication, as appropriate). The hardware elements can include one or more processors 810, including without limitation one or more general-purpose processors and / or one or more special-purpose processors (such as digital signal processing chips, graphics acceleration chips, and / or the like); one or more input devices 815, which can include without limitation a mouse, a keyboard, a touchscreen, and / or the like; and one or more output devices 820, which can include without limitation a display device, a printer, and / or the like. ​

[0068] Computer system 800 can further include one or more storage devices 825 (and / or communication therewith), which can include, without limitation, local and / or network accessible storage, and / or can include, without limitation, a disk drive, a drive array, an optical storage device, a solid-state storage device such as a random access memory ("RAM") and / or a read-only memory ("ROM"), which can be programmable, flash- updateable, and / or the like. Computer system 800 can also include a communication subsystem 830, which can include, without limitation, a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device and / or chipset (such as 802.11 device, WiFi device, WiMax device, cellular communication facility, etc.), and / or the like. Communication subsystem 830 can permit data to be exchanged with a network (such as the network described below, as an example) and / or any other device described herein. In many embodiments, computer system 800 will further include a working memory 835, which can include a RAM or ROM device, as described above.

[0069] Computer system 800 can also include software elements, shown as being currently located within working memory 835, including an operating system 840 and / or other code (such as one or more application programs 845), which can include computer programs to implement methodologies described herein and / or designed to implement methods of the present application and / or to configure systems of the present application, as described herein. By way of example, one or more processes described with reference to the methods discussed above can be implemented as code and / or instructions executable by a computer (and / or a processor within a computer). Sets of these instructions and / or code can be stored in a computer-readable storage medium, such as the storage device(s) 825 described above. In some cases, the storage medium can be incorporated within a computer system, such as system 800. In other embodiments, the storage medium can be separate from a computer system (i.e., a removable medium, such as a compact disc, etc.), and / or provided in an installation package, such that the storage medium can be used to program a general purpose computer with the instructions / code stored thereon. These instructions can take the form of executable code which is executable by computer system 800, and / or can take the form of source code and / or installable code which, upon compilation and / or installation on computer system 800 (e.g., using any of a variety of generally available compilers, installation programs, compression / decompression utilities, etc.), then takes the form of executable code. In certain embodiments, this can include a page fetcher function, and can include a separate page fetcher module as part of this page fetcher function, which is executed by a process that allows a web page to be displayed on output device 820 of such computer system 800.

[0070] It will be apparent to those skilled in the art that substantial variations can be made in accordance with specific requirements. For example, customized hardware might also be used, and / or particular elements might be implemented in hardware, software (including portable software, such as applets, etc.), or both. Further, connection to other computing devices such as network input / output devices can be employed.

[0071] In one aspect, the application employs a computer system (such as the computer system 800) to perform methods of the application. Depending on the embodiment, the computer system 800 can respond to various inputs (e.g., instructions) from the processor 810 executing one or more sequences of one or more instructions contained in the working memory 835, which can be incorporated into an operating system 840 and / or other code, such as an application program 845. Such instructions can be read into the working memory 835 from another machine-readable medium, such as one or more storage device(s) 825. The sequences of instructions contained in the working memory 835 can cause the processor 810 to perform one or more processes of the methods described herein, only as examples.

[0072] The terms "machine-readable medium" and "computer-readable medium," as used as herein, refer to any medium that participates in providing data that causes a machine to operate in a specific fashion. In an embodiment implemented using the computer system 800, various machine-readable media might be involved in storing and / or carrying the sequences of instructions to the processor 810 for execution. In many implementations, a computer-readable medium is a physical and / or tangible storage medium. Such a medium can take many forms, including but not limited to, non-volatile media, volatile media, or any other available medium that is capable of storing and / or carrying sequences of instructions that execute on the processor 810. Non-volatile media includes, for example, optical or magnetic disks, such as the storage device(s) 825. Volatile media includes, for example, dynamic memory, such as the working memory 835. Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, solid-state drive, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, any other physical and / or tangible medium, or any other medium that is capable of storing and / or carrying the sequences of instructions as desired by the computer system 800. The sequences of instructions can be carried by the system bus 830 to the processor 810 to execute various processes defined by the sequences of instructions. The various processes can then be executed by the processor 810, specifically, by one or more of the execution units 830A-830N.

[0073] The communications subsystem 830 (and / or components thereof) typically will receive signals, and the bus 805 then carries the signals (and / or the data, instructions, requests, etc. carried by the signals) to the working memory 835, from which the processor 810 retrieves and executes the instructions. The instructions received by the working memory 835 can optionally be stored on a storage device 825 either before or after execution by the processor 810.

[0074] Embodiment groups include systems for implementing time-dependent cues. Figure 9 A schematic diagram of a network system 900 that can be used in accordance with a group of embodiments is shown. In such embodiments, the network system 900 can represent any number of client and server devices that enable content to be delivered from a content provider to a user system as part of a web transaction across a network. The system 900 can include one or more user computers 905, such as the computers of the client machines 110 used to implement the methods of the present application. The user computers 905 can be general purpose personal computers and / or can have any of a variety of applications in common, including one or more applications configured to perform methods of the present application, as well as one or more office applications, database client and / or server applications, and page fetcher applications. Alternatively, the user computers 905 can be any other electronic device, such as a thin-client machine, a mobile telephone using the Internet, a tablet computer, a phablet, a wearable device, a device using the Internet, and / or a personal digital assistant (PDA), that is capable of communicating via a network, such as the network 910 described below, and / or displaying and navigating web pages or other types of electronic documents. Although the example system 900 is shown with three user computers 905a-c, any number of user computers can be supported. Figure 1

[0075] Certain embodiments of the present application operate in a networked environment, which can include a network 910, such as the network 140 used to implement the Figure 1 The network 910 can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially-available protocols, including without limitation TCP / IP, UDP / IP, SNA, IPX, etc. Merely by way of example, the network 910 can be a local area network ("LAN"), including without limitation an Ethernet network, a Token-Ring network, and / or the like; a wide-area network ("WAN"); a virtual network, including without limitation a virtual private network ("VPN"); the Internet; an intranet; an extranet; a public switched telephone network ("PSTN"); an infrared network; a wireless network, including without limitation a network operating under any of the IEEE 802.11 suite of protocols or any other wireless protocol; and / or any combination of these and / or other networks.

[0076] ​Embodiments of the application can include one or more servers 915 (or similar types of computing systems). Embodiments of the server 915 can be used to implement the content server 120 and / or the hint machine 130. As noted above, the hint machine 130 can also be part of one or more content servers 120 and / or part of one or more client machines 110, which can be implemented as one or more servers 915. Each server 915 can be configured with an operating system, including but not limited to any of the operating systems discussed above, as well as any commercially- (or freely-) available server operating systems. Each server 915 can also run one or more applications, which can be configured to serve content to one or more user computers 905 and / or other servers 915.

[0077] By way of example only, one of the servers 915 can be a web server, which can be used to handle requests for web pages or other electronic documents from the user computers 905, by way of example only. The web server can also run various server applications, including an HTTP server, an FTP server, a CGI server, a database server, a script server, etc. In some embodiments of the application, the web server can be configured to serve web pages that can operate within a page extractor on one or more user computers 905 to perform the methods of the application. In certain embodiments, the servers 915 can also include CDN devices that provide similar or identical content from alternate servers as the primary content servers that are the source of particular objects in web transactions.

[0078] In some embodiments, server(s) 915 can include one or more application servers, which can include one or more applications accessible by clients running on user computer(s) 905 and / or other server(s) 915. By way of example, the server(s) 915 can be one or more general purpose computing devices, such as a network application (which in some cases can be configured to perform the methods of the present application) that can respond to requests from user computer(s) 905 and / or other server(s) 915, including, without limitation, web browsers, database servers, and the like. By way of example, the network application can be implemented as one or more scripts or programs written in any appropriate programming language such as, for example, Java, C, C++, C#, or the like. The application server(s) can also include database servers, which can handle requests from clients running on user computer(s) 905 and / or another server 915 (depending on the configuration, including database clients, API clients, page scrapers, and the like). In some embodiments, in accordance with embodiments of the present application, the application server(s) can dynamically create web pages for displaying information, such as information displayed as part of the various page scrapers discussed herein. Data provided by the application server(s) can be formatted as web pages (e.g., including HTML, scripts, and the like) and / or can be forwarded to user computer(s) 905 via content server(s) 120. Similarly, content server(s) 120 can receive web page requests and / or input data from user computer(s) 905 and / or forward web page requests and / or input data to the application server(s). In some cases, content server(s) 120 can be integrated with the application server(s).

[0079] According to further embodiments, one or more server(s) 915 can function as a file server and / or can include one or more files (e.g., application code, data files, etc.) necessary to implement the methods of the present application, which are incorporated by an application running on user computer(s) 905 and / or another server 915. Alternatively, as will be appreciated by those skilled in the art, a file server can include all necessary files, allowing the user computer(s) 905 and / or server(s) 915 to call such applications remotely, as needed. It should be noted that the functions described with reference to the various servers herein (e.g., application servers, database servers, web servers, file servers, etc.) can be performed by a single server and / or a plurality of specialized servers, depending on the needs and parameters of the particular implementation.

[0080] In certain embodiments, the system can include one or more databases 920, such as for implementing resource sample data store 280, prompt information data store 135, and / or any other suitable data store used by the systems and methods of the various embodiments. Such databases can include storage of network transaction histories and derived prompts from the network transaction histories. Any supporting details related to the creation of such prompts can additionally be stored in such databases. The location of databases 920 is arbitrary: by way of example only, database 920a can reside on storage media located at (and / or resident to) server 915a (and / or user computer 905). Alternatively, database 920b can be remote to any or all of computers 905 or servers 915a, b, so long as database 920b can communicate with one or more of these (e.g., via network 910). In a particular embodiment set, databases 920 can reside in a storage area network ("SAN") familiar to those skilled in the art. Similarly, any necessary files for performing the functions attributed to the computers 905 or servers 915 can be stored locally and / or remotely, as appropriate. In embodiment sets, databases 920 can be a relational database adapted to store, update, and retrieve data in response to SQL-formatted commands. For example, the databases can be controlled and / or maintained by a database server, as described above.

[0081] In any of the embodiments described herein, the Hypertext Transfer Protocol Secure (HTTPS) can also be used to implement any of the communications, Hypertext Protocol (HTTP) requests or responses, web transactions, portions of web transactions, or any combination of any communications that can use HTTP in whole or in part. The use of HTTPS involves encryption and various security measures, and any of the embodiments described herein can be adapted to implement the systems and methods described herein, thereby mediating the encryption and security associated with HTTPS in a variety of ways.

[0082] In certain embodiments, the proxy devices involved in any of the communications described herein can be provided with details associated with HTTPS communications in order to implement any of the proxy device functionality described herein. These details can be provided by the user device with the user's consent to allow the proxy device or prompt machine device to interact with the user's communications. This can enable the proxy to decrypt HTTPS communications to allow any of the functionality described herein. Such decryption-allowing details can also be provided by the content server or content server operator to enable the proxy device to interact with the content provider's communications. Certain embodiments can function in the same manner as described above for the proxy devices for aspects of the embodiments described herein related to client-side functionality or page scraper functionality. This functionality can also be used for any other server-side or content devices.

[0083] In other embodiments, in addition to proxy decryption of HTTPS communications, a browser on a user device can interact with a proxy device or hint machine device independently of the original HTTPS communications to provide details of the HTTPS communications to the proxy device or hint machine device. In a user device page extractor, this can involve using page extractor modules, page extractor plugins, or page extractors that have specialized core functionality to observe HTTPS communications before they are encrypted. If needed, the independent communications with the proxy device or hint machine device can then be additionally secured with security similar to that used for the original HTTPS communications. For embodiments described herein that can involve integration with a content provider system or reverse proxy system, HTTPS communications received at the content provider system or reverse proxy can be used as part of the embodiments described herein after the content provider system or reverse proxy decrypts the HTTPS communications. Thus, any "observation" or "identification" functionality described herein can be performed using HTTPS communications for the client machine, proxy device, and content provider device. Similarly, any embodiments described herein can be implemented with other secure connections and communication systems or protocols in addition to HTTPS described above.

[0084] Certain embodiments described herein involve a proxy device or proxy system. As described herein, a proxy device can be one or more devices that operate as part of a system to intercept communications between a user device and a content device. Such a proxy can be transparent to the user and content device, or can be part of a multi-step communication path that is apparent to the user and content device. In certain embodiments, a proxy device can operate as part of a forward proxy system, where communications from a group of user devices are all transmitted to a wider network (such as the Internet) via the forward proxy system. In alternative embodiments, a proxy device can operate as part of a reverse proxy system, where the reverse proxy system operates as a path of access to a plurality of content server devices. It will be apparent that embodiments described herein as including a proxy will apply to both forward proxy systems and reverse proxy systems, as well as any configuration of a system that includes a proxy, with appropriate adjustments for the particular system architecture.

[0085] The methods disclosed herein comprise one or more actions for implementing the methods. The methods and / or actions can be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions can be modified without departing from the scope of the claims.

[0086] The functions can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions on a tangible computer-readable medium. A storage medium can be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks and and disc, wherein disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0087] A computer program product can perform certain operations presented herein. For example, such a computer program product can be a computer readable tangible medium having instructions stored (and / or encoded) thereon, the instructions being executable by one or more processors to perform the operations described herein. A computer program product can include packaging material. Software or instructions can also be transmitted over a transmission medium. For example, software can be transmitted from a website, server, or other remote source using a transmission medium, such as a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or the like, or using wireless technologies, such as infrared, radio, and microwave, among others.

[0088] Moreover, modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by a user terminal and / or base station as applicable, such as from the server and / or over-the-air. Alternatively, various methods described herein can be provided via a storage means (e.g., RAM, ROM, a physical storage medium such as a CD or floppy disk, etc.), such that a user terminal and / or base station can obtain the various methods upon coupling or providing the storage means to the device.

[0089] In describing the present application, the following terminology will be used: The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to an item includes a reference to one or more items. The terms "one" or "one(s)" mean one, two, or more and generally apply to a selection or all of a quantity. The term "plurality" means two or more items. The term "about" means amounts, sizes, lengths, sizes, formulations, parameters, shapes and other characteristics that are not, and should not, be exact, but can be approximated with the degree of accuracy that enables the desired function, while being economically feasible. The term "substantially" means that the recited characteristic, parameter, or value need not be achieved exactly, but that deviations or variations, including for example, tolerances, measurement error, measurement accuracy limitations and other factors known to those of skill in the art can occur. Numerical data can be expressed or presented herein in a range format. It is to be understood that such a range format is used merely for convenience and brevity and thus should be interpreted flexibly to include not only the numerical values explicitly recited as the limits of the range, but also to include all the individual numerical values or sub-ranges within that range as if each numerical value and sub-range is explicitly recited. As an illustration, a numerical range of "about 1 to 5" should be interpreted to include not only the explicitly recited values of about 1 to about 5, but also include individual values and sub-ranges within the indicated range, such as 2, 3, and 4, and 1-3, 2-4, and 3-5, etc. The same principle applies to ranges reciting only one numerical value (e.g., "greater than about 1") and should apply regardless of the width of the range or the characteristics being described. For convenience, multiple items can be presented in a common list. However, these lists should be construed as though each member of the list is individually identified as a separate and unique member. Thus, no individual member of such list should be construed as a de facto equivalent of any other member of the same list. Furthermore, when the term "and / or" is used in the discussion of certain embodiments of the application, it will be understood that the term is used in the same sense as the phrase "and / or" is used in the common dictionary. The term "alternatively" means selecting one of two or more alternatives, and is not intended to limit the selection to only the listed alternatives or to only one of the listed alternatives at a time, unless the context clearly indicates otherwise. The term "coupled" as used herein does not require that components be directly connected to one another. Rather, the term is intended to also include configurations in which one or more other components are included between the coupled components. Such other components can include amplifiers, attenuators, isolators, directional couplers, redundant switches, and similar components, for example.Further, as used herein, including in the claims, "or" as used in a list of items prefaced by "at least one of indicates a disjunctive list such that, for example, a list stating "at least one of A, B, or C" means A or B or C or AB or AC or BC or ABC (i.e., A and B and C). Further, the term "exemplary" is not meant to connote that an example better or better than other examples. Similarly, the term "optimization" is not intended to limit the embodiments to optimal methods or results, but rather is intended to express a seeking towards a most optimal result (e.g., improvement, etc.). As used herein, a "set" of elements is intended to denote "one or more" of those elements, unless expressly required to have more than one or expressly allowed to be an empty set.

[0090] Various changes, substitutions and alterations can be made to the techniques described herein without departing from the technology defined by the appended claims. Moreover, the scope of the disclosure and the claims is not limited to specific aspects of the processes, machines, manufactures, compositions of matter, means, methods and steps described herein. Processes, machines, manufactures, compositions of matter, means, methods and steps that perform substantially the same function or achieve substantially the same result as those described herein are within the scope of the claims. Accordingly, the appended claims are intended to cover within their scope all processes, machines, manufactures, compositions of matter, means, methods and steps substantially as described herein.

Claims

1. A hint generation method, the method comprising: receiving, by a hint machine from one or more client machines, feedback information generated by the one or more client machines based on previous renderings of a web page, wherein: the previous renderings of the web page include obtaining a root object of the web page and one or more sub-resources, the feedback information includes a plurality of resource samples of one or more of the one or more sub-resources, each resource sample includes a sampling time corresponding to one of the one or more sub-resources and indicates a rendering status of the corresponding sub-resource during a corresponding previous rendering of the web page at the respective sampling time, the rendering status indicating whether the corresponding sub-resource was loaded at the corresponding sampling time; determining resource relationships between the plurality of resource samples, the resource relationships indicating a correspondence between a rendering status of a first resource sample and a rendering status of a second resource sample; determining, by the hint machine, a time-based probability function from the plurality of resource samples, the time-based probability function generating a probability that a single sub-resource of the one or more sub-resources will be requested and a second sub-resource will be requested based on the determined resource relationships to the single sub-resource in a process of rendering the web page, the time-based probability function being determined by applying a weighting function to each resource sample of the plurality of resource samples to weight an influence of each resource sample on the probability generated by the time-based probability function for the single sub-resource; computing, by the hint machine, one or more probabilities using the time-based probability function, each probability corresponding to one of the one or more sub-resources and indicating a probability that the corresponding sub-resource of the one or more sub-resources will be requested from a content server in a process of rendering the web page at a future particular time; and generating, by the hint machine, a hint from the probabilities for pre-fetching one or more of the one or more sub-resources as part of a future rendering of the web page by a requesting client machine.

2. The method of claim 1, further comprising: receiving, at the hint machine from the requesting client machine at a request time, a hint request associated with a rendering of the web page by the requesting client machine; generating, by the hint machine in response to the hint request, a hint response including the hint; and transmitting the hint response from the hint machine to the requesting client machine.

3. The method of claim 2, wherein: generating the hint includes computing, using the time-based probability function, an instantaneous hint probability of the hint at the request time or a specified time after the request time and a corresponding probability of the sub-resource; and transmitting the hint response includes transmitting the hint with the instantaneous hint probability.

4. The method of claim 2, wherein: transmitting the hint response includes transmitting the hint with the time-based probability function.

5. The method of claim 1, further comprising applying a hint decay model to the one or more probabilities computed using the time-based probability function.

6. The method of claim 2, wherein: the hint request indicates a root uniform resource locator (URL) of the web page; and the sub-resource is a sub-URL of the web page.

7. The method of claim 2, wherein, the one or more probabilities are computed prior to receiving the hint request.

8. The method of claim 1, wherein, computing the one or more probabilities includes classifying each resource sample according to a presence model.

9. The method of claim 1, wherein, at least some of the resource samples are received by the hint machine from the one or more client machines over a communication network after the web page is rendered at a respective client machine.

10. The method of claim 1, wherein, at least some of the resource samples are received by the hint machine as part of an automated web crawl.

11. The method of claim 1, wherein, the hint machine is co-resident with the requesting client machine.

12. The method of claim 1, wherein, computing the one or more probabilities from the plurality of resource samples includes: determining a freshness of each resource sample; and weighting an influence of each resource sample on a corresponding probability according to the freshness of the resource sample.

13. The method of claim 1, wherein, the time-based probability function includes a function that yields a probability that a corresponding sub-resource will be used as part of a future rendering of the web page, the probability being a function of a time associated with the future rendering of the web page.

14. The method of claim 1, further comprising: generating a model for the web page based on the feedback information, the model indicating one or more probabilities of the one or more sub-resources, the one or more probabilities indicating whether a respective sub-resource will be used to render the web page at some future time, and wherein generating the probabilities is based on the model.

15. A hint generation system, the system comprising: a hint machine disposed in a communication network, the hint machine comprising: a sample data store; a resource sampler, the resource sampler operative to receive, from one or more client machines, feedback information generated by the one or more client machines based on prior renderings of a web page, wherein rendering the web page includes obtaining a root object and one or more sub-resources, the feedback information including a plurality of resource samples for one or more of the one or more sub-resources of the web page, each resource sample including a sampling time corresponding to one of the one or more sub-resources and indicating a rendering state of the respective sub-resource during a corresponding prior rendering of the web page at the respective sampling time, the rendering state indicating whether the respective sub-resource was loaded at the corresponding sampling time; a hint generator, the hint generator operative to: determine resource relationships among the plurality of resource samples, the resource relationships indicating a correspondence between a rendering state of a first resource sample and a rendering state of a second resource sample; determining a time-based probability function from the plurality of resource samples, the time-based probability function generating a probability that a single sub-resource of the one or more sub-resources will be requested during the course of rendering the web page and based on the determined resource relationships to the single sub-resource will request a second sub-resource, the time-based probability function determined by applying a weighting function to each resource sample in the plurality of resource samples to weight an influence of each resource sample on the probability generated by the time-based probability function for the single sub-resource; computing one or more probabilities using the time-based probability function, each probability corresponding to one of the one or more sub-resources and indicating a probability that the corresponding sub-resource of the one or more sub-resources will be requested from the content server in the future at a particular time during the course of rendering the web page; and generating a hint from the probabilities for pre-fetching one or more of the one or more sub-resources as part of a request client machine rendering the web page in the future; and a communication subsystem operative to communicate the hint to the request client machine.

16. The system of claim 15, wherein: the communication subsystem is further operative to receive a hint request from the request client machine over the communication network at a request time associated with the request client machine rendering the web page; the hint generator is further operative to generate a hint response containing the hint in response to the hint request; and the communication subsystem is further operative to communicate the hint response to the request client machine.

17. The system of claim 16, wherein: the hint generator is operative to compute one or more probabilities of the hint by using the time-based probability function to compute an instantaneous hint probability of the hint at the request time or at a specified time after the request time and the probability of the corresponding sub-resource; and the communication subsystem is operative to communicate the hint response by communicating the hint with the instantaneous hint probability.

18. The system of claim 16, wherein: the communication subsystem is operative to communicate the hint response by communicating the hint with the time-based probability function.

19. The system of claim 15, wherein, the hint generator is further operative to apply a hint decay model to adjust one or more probabilities computed using the time-based probability function.

20. The system of claim 18, further comprising: a first client machine in communication with the hint machine over the communication network, the first client machine comprising: a client hint subsystem operative to receive the hint from the hint machine over the communication network; and a page fetcher operative to pre-fetch a sub-resource in association with rendering the web page and in accordance with the received hint.

21. The system of claim 20, wherein, the first client machine further comprising: a hint store operative to store the received hint and the hint probability function; and the hint generator is further operative to apply a hint decay model to adjust one or more probabilities computed using the time-based probability function. wherein the page extractor is operative to pre-fetch by calculating a client-side hint for pre-fetching the sub-resource according to the request time and the stored hint probability function, and pre-fetching the associated sub-resource according to the client-side hint.

22. The system of claim 21, wherein: the page extractor is further operative to determine whether an updated hint is available from the hint machine at the request time; and the page extractor is operative to calculate the client-side hint in response to the determination.

23. The system of claim 15, further comprising: a first client machine comprising: the hint machine; a client hint subsystem operative to receive the hint from the hint machine; and a page extractor operative to pre-fetch a sub-resource in association with rendering the web page and according to the received hint.

24. The system of claim 15, wherein, the resource sampler is further operative to receive: one of the resource samples from the first client machine associated with a previous rendering of the web page by the first client machine, and another one of the resource samples from a second client machine associated with a previous rendering of the web page by the second client machine; and wherein the second client machine is different from the first client machine.

25. The system of claim 15, wherein, the time-based probability function comprises a function that yields a probability that a corresponding sub-resource will be used as part of a future rendering of the web page, the probability being a function of a time associated with the future rendering of the web page.

26. A hint generation method, the method comprising: receiving, at a client machine from a hint machine over a communication network, a plurality of hints for pre-fetching one or more sub-resources in association with loading a web page, the hints being generated by the hint machine by using a time-based probability function determined according to a plurality of resource samples and by calculating one or more probabilities using the time-based probability function, the time-based probability function generating a probability that a single sub-resource of the one or more sub-resources will be requested during a rendering of the web page in the future and a second sub-resource will be requested based on a determined resource relationship of the single sub-resource, each probability corresponding to one of the one or more sub-resources and indicating a probability that the corresponding sub-resource of the one or more sub-resources will be requested from a content server during a rendering of the web page at a particular time in the future, each resource sample being received by the hint machine with respect to a corresponding sampling time and indicating a rendering state of a corresponding resource with respect to rendering the web page at the corresponding sampling time, the rendering state indicating whether the corresponding sub-resource was loaded at the corresponding sampling time, and the time-based probability function being determined by applying a weighting function to each resource sample of the plurality of resource samples to weight an influence of each resource sample on the probability generated by the time-based probability function for the single sub-resource; and pre-fetching at least some of the one or more sub-resources in association with loading the web page according to the received plurality of hints.

27. The method of claim 26, further comprising: storing the hints on the client machine, the hints including a hint probability function computed by the hinting machine to indicate a probability over time that the resource will be requested at some future time as part of loading the web page, wherein the pre-fetching includes computing a set of client-side hints for pre-fetching an associated set of resources according to the request time of the web page and the stored hint probability function, and pre-fetching the associated set of resources according to the set of client-side hints.

28. The method of claim 27, further comprising: determining, by the client machine, that updated hints cannot be obtained from the hinting machine at the request time, wherein computing the set of client-side hints is performed in response to the determining.

Images