A methodology for detecting public web page's content change from the last time any user accessed and index the content in near real-time
A JavaScript-based method for detecting webpage content changes using hashes addresses inefficiencies in existing methods, enabling near real-time indexing with reduced complexity and user effort.
Patent Information
- Authority / Receiving Office
- GB · GB
- Patent Type
- Applications
- Current Assignee / Owner
- ARUMILLI KANTI KALYAN
- Filing Date
- 2023-05-27
- Publication Date
- 2026-04-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing methods for detecting and indexing content changes on web pages are inefficient and require complex SDKs or frequent crawling, making them unsuitable for near real-time updates.
A JavaScript-based approach that generates hashes of webpage content and compares them with server-stored hashes to detect changes, allowing near real-time indexing without complex SDKs or frequent crawling, using a secret key and hash check.
Enables seamless, near real-time detection and indexing of webpage content changes with minimal user effort, maintaining competitiveness by reducing integration complexity.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
An innovative way for detecting webpage content changes and indexing documents. My Startup's SaaS product offers Search as a Service. Customers would include a javascript file provided. The SaaS product needs to identify content changes using several methods and index in almost realtime. Other normal methods include 1. If programmatic / SDK, the SDK would send a request to the server for re-indexing. 2. Crawler looks for RSS / SiteMap updates once very few (pre-defined) minutes such as 5, 10, 15, 60. 3. Pingbacks (for blogs) 4. IndexNow protocol WebVeta proposes and plans to use a slightly innovative way of determining near real-time changes by using the included javascript for generating hash and detecting changes, then notifying the server of changes. Pretty much when a new webpage has been added or updated, the SaaS product would be notified on first access in a normal web browser without any need for complex SDKs. The content updater (end user - owner of website), updates the website and views the modified web page using normal web browser. Or any other random user views the modified website. Because the javascript provided by WebVeta would be on the page, the javascript apart from showing the search widget, results etc... also performs a hash check of the main content (without the head, aside, nav, footer i.e looking for actual content changes) of the page and sends the hash to WebVeta's web server. Or the other way around I.e the javascript pre-fetches the server's copy of the hash and compares with the generated hash. If the hash is different, WebVeta server responds back with a "no match" or the javascript sends a request to the server. The javascipt can pass the HTML content to the server for indexing, or WebVeta can crawl / re-crawl the web page. Another possibility is for the admin / content editor to view the webpage and include a special parameter in the querystring such as ?webveta-reindex=xxxxxxxxxxx. I.e xxxxx would be a long secret, the javascript from WebVeta embedded in the webpage looks for the webveta-reindex, its value, the Javascript sends a request to the browser to re-index and passes the secret key. The server processes (I.e re-indexes) if the secret key matches. The above two methodologies can be combined I.e secret key + hash check. If completely new webpage or if the webpage has not been previously indexed by WebVeta, the hash stored on the server would be null Le server's side of hash would be equivalent to empty string. This way WebVeta can stay competitive by allowing near real-time indexing of publicly available content of websites with minimal effort by end-users for integration l.e seamless integration.
Claims
1) Javascript based content hash checking for detecting changes in a web page’s content.