Client-Side Web Search Agent for Real-Time Content Crawling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing web search services rely on centralized databases that cannot track updates to the web in real-time, leading to outdated search results, especially with the proliferation of dynamically generated websites, and introduce subjective bias through popularity algorithms.
Innovation Solution
A client-side searching method that identifies relevant search origins, crawls content in real-time, and returns search results based on user queries, utilizing a graphical user interface application to retrieve and review content from network locations, submission fields, and dynamically generated content, prioritizing relevance and updating results continuously.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If centralized databases are used to index web pages, then search coverage is improved, but real-time update capability deteriorates
Solution Approach 1:
The patent divides the centralized indexing system into distributed client-side components. Each user's computer runs a local search agent that independently crawls and indexes web pages, eliminating the single-point bottleneck of centralized databases and enabling simultaneous real-time updates across multiple distributed nodes.
Solution Approach 2:
The search system performs self-updating through automated client-side agents that continuously crawl web pages, extract content, and maintain local indexes without requiring manual intervention or centralized coordination. Each client independently manages its own search index, enabling autonomous real-time updates.
2Reliability
If web-crawlers continuously crawl the Internet to update indexes, then currency of search results is improved, but computational resources required deteriorates
Solution Approach 1:
The patent distributes the computational burden of web crawling across numerous client-side agents rather than concentrating it in centralized server farms. Each client agent handles a portion of the crawling workload, dividing the immense computational resources requirement into manageable segments that can be executed on ordinary personal computers.
Solution Approach 2:
User computers automatically perform crawling and indexing operations in the background without requiring dedicated server infrastructure. The system utilizes idle computational resources of end-user machines to maintain current search indexes, eliminating the need for expensive centralized computing facilities.
3Measurement precision
If link popularity algorithms are used to order search results, then relevance determination is improved, but subjectivity and bias introduced deteriorates
Solution Approach 1:
Instead of using link popularity to determine relevance (how many other pages link to this page), the patent inverts the approach by examining how many links this page contains to other relevant pages. This reverse link analysis reduces bias toward popular sites and highlights pages that actively connect to relevant content, providing a more objective relevance measure.
Data Source
AI summary
In a preferred form, the present invention provides a method (200) for searching the Internet (14n). The method (200) comprising the steps of: in response to a user search query, identifying at least one search origin (12n) comprising an Internet resource (12n) deemed of relevance; retrieving and reviewing the contents of the search origin (12n) to identify the presence of one or more search avenues (12n) stemming from the search origin (12n), identifiable search avenues comprising: sites (12n) which are ascertained by the search origin contents (12n); submission fields presented by the search origin (12n); and/or, dynamically generated content (12n) retrieved from the search origin, and, perusing identified search avenues (12n) to identify search results to be returned in response to the user search query.