Selective Web Crawling for Real-Time Image Scoring Memory Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in managing computer memory and processing resources when generating scores for images on web pages, particularly when storing and updating databases for multiple domains, leading to increased memory and processing demands.
Innovation Solution
A system that stores pointers to web pages instead of images, allowing real-time performance scoring by accessing and scraping images from third-party servers, reducing memory usage and maintaining database efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the computer stores images for all web pages across multiple domains in a database, then the ability to generate real-time performance scores for images is improved, but the memory resources required increase quickly
Solution Approach 1:
The patent creates a database that stores copies of image URLs (pointers) rather than the actual image files. This allows the system to maintain a comprehensive database for scoring purposes while consuming minimal memory resources. The database contains only the essential information (image locations) needed to retrieve and score images on-demand.
Solution Approach 2:
The patent introduces an intermediary database layer that stores pointers to images hosted on third-party servers. This intermediary structure enables the scoring system to access images without storing them locally, thereby maintaining real-time scoring capability while avoiding the memory overhead of storing actual image data.
2Measurement precision
If the computer continuously communicates with website servers to update the database with added or removed images, then the real-time accuracy of performance scores is improved, but the processing resources required increase
Solution Approach 1:
The patent implements periodic crawling of web pages to update the image database. Instead of continuous real-time monitoring, the system periodically visits web pages to detect newly added or removed images. This periodic approach maintains database accuracy while significantly reducing processing resource consumption compared to continuous monitoring.
Solution Approach 2:
The crawling system automatically detects changes on web pages and updates the database without requiring manual intervention or continuous external triggers. The system serves itself by periodically checking for changes and maintaining its own data accuracy, reducing the need for additional processing resources.
3Productivity
If a large number of content items are displayed on a webpage to increase target audience interaction, then the interaction rate is improved, but the network bandwidth and computer memory are exceeded causing latency
Solution Approach 1:
The patent uses a database of image pointers that can be quickly retrieved and used to generate web pages with multiple content items. This allows the system to display a large number of images without the bandwidth overhead of storing and transferring actual image data, thereby maintaining fast page loading speeds while achieving high interaction rates.
Data Source
AI summary
A method includes storing a database comprising a plurality of pointers to web pages and identifiers of entities associated with the plurality of pointers; receiving a first request comprising a first identifier; identifying subset of the plurality of pointers from the database responsive to each pointer of the subset having a stored association with a first identification that matches the first identifier; responsive to identifying the subset of the plurality of pointers, establishing, via one or more pointers, a connection with a server hosting a set of web pages associated with the subset of the plurality of pointers; retrieving one or more images or videos from each of the set of web pages over the established connection; calculating a performance score for each of the one or more images or videos; and generating a record identifying the performance score for each of the one or more images or videos.