Resource Pre-fetch Using Age Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for accessing webpages often result in latency due to inefficient pre-fetching of resources, leading to slow webpage navigation and bandwidth wastage, as they struggle to accurately assess which resources to pre-fetch for reliable improvements in response time.
Innovation Solution
A computing system that crawls multiple webpages to identify resources, determines their age and pre-fetch measure based on usage information, and generates pre-fetch index entries for resources that meet freshness and stability criteria, allowing for efficient pre-fetching of resources that are likely to be accessed by users.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If resources are pre-fetched without age threshold criteria, then response time may be improved, but bandwidth is wasted on fetching stale or frequently changing resources
Solution Approach 1:
The system performs preliminary actions by pre-fetching resources before they are actually needed, but only after determining they meet stability criteria. The crawler identifies resources and determines their age and stability characteristics in advance, then pre-fetches only those resources that are unlikely to change, thereby reducing response time without wasting bandwidth on volatile resources.
Solution Approach 2:
The system changes the parameter of resource selection by introducing age threshold and stability criteria. Instead of pre-fetching all resources or using simple heuristics, the system evaluates resources based on their age (comparing crawl timestamps) and stability characteristics, making pre-fetching decisions based on these transformed parameters that indicate resource reliability.
2Speed
If all resources are pre-fetched, then webpage loading speed improves, but system bandwidth and processor utilization increase unnecessarily
Solution Approach 1:
The system applies local quality by treating different resources differently based on their individual characteristics. Instead of a uniform pre-fetching approach, the system evaluates each resource's age and stability independently, applying pre-fetching only to resources that meet specific quality criteria (stable, non-frequently-changing resources), thereby optimizing bandwidth usage while maintaining loading speed benefits.
Solution Approach 2:
The system performs partial action by selecting only a subset of resources for pre-fetching rather than all resources. By using age thresholds and stability criteria to identify a specific portion of resources that are suitable for pre-fetching, the system achieves the beneficial effect of faster loading without the excessive cost of prefetching every possible resource.
3Reliability
If the pre-fetch index is rebuilt frequently to maintain accuracy, then resource freshness improves, but system overhead and processing time increase
Solution Approach 1:
The system implements periodic action by rebuilding the pre-fetch index at scheduled intervals rather than continuously or too frequently. The crawler periodically crawls webpages and updates resource age information, determining when index rebuilding is necessary based on these periodic assessments, thereby maintaining resource freshness while controlling system overhead through rhythmical rather than constant operations.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying a set of resources in response to crawling multiple webpages that use at least one resource in the set. For each resource in the set, a system determines an age of the resource using a timestamp for the resource. The system determines a pre-fetch measure of the resource based on the age of the resource and usage information that describes use of the resource at a webpage. The system selects a first resource from the set based on the pre-fetch measure and determines whether a respective age of the selected first resource exceeds a threshold age. The system generates an index entry for a pre-fetch index. The index entry includes a command to pre-fetch the first resource based on a determination that the respective age of the first resource exceeds the threshold age.