Web Retrieval Script Reuse and Repair for Changing Page Layouts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for automated web site information retrieval require frequent updates due to site layout changes, leading to increased costs and user frustration, especially when many sites change layouts simultaneously, and pose challenges in justifying resource allocation for scripts that serve few users.
Innovation Solution
A system and method that allows information retrieval from web sites by combining elements from different user-provided scripts, using a plug-in or proxy server to assist users in identifying necessary fields on a web page, and automatically repairing or replacing scripts that fail, ensuring continuous data availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a dedicated programmer writes a script for each web site, then information retrieval can be automated, but the cost increases significantly when many web sites are involved and site layouts change frequently
Solution Approach 1:
The system enables end-users to create their own scripts by demonstrating operations on web sites, eliminating the need for dedicated programmers. Users interact with a browser plug-in to define field locations and retrieve information, making the system self-serve its own script creation needs without requiring professional programming resources.
Solution Approach 2:
The system captures user interactions with web sites through browser plug-ins, recording mouse movements, clicks, and navigation paths to generate scripts. These scripts are copied from user demonstrations and stored for automated information retrieval, allowing the system to replicate user operations automatically.
2Reliability
If a team of programmers updates scripts frequently to accommodate web site layout changes, then information retrieval reliability is maintained, but user wait times increase and user frustration grows
Solution Approach 1:
The system continuously monitors script performance and web site changes, using feedback loops to detect when scripts become obsolete. When layout changes occur, the system automatically identifies affected scripts and triggers updates by capturing new user demonstrations, ensuring reliable information retrieval without manual intervention delays.
Solution Approach 2:
The system proactively captures and stores user demonstrations of web site operations, preparing scripts in advance before they are needed for automated retrieval. This preliminary action ensures that when information retrieval is required, the scripts are already available and up-to-date, eliminating wait times associated with manual script creation.
3Adaptability or versatility
If resources are allocated to write scripts for web sites with few users, then complete information coverage is achieved, but the cost per user increases and resource efficiency decreases
Solution Approach 1:
The system dynamically adjusts script creation based on actual user needs and web site accessibility. Instead of maintaining fixed scripts for all web sites, the system actively captures demonstrations only when needed, adapting its behavior to real-time requirements and eliminating unnecessary resource allocation for low-value web sites.
Solution Approach 2:
The system creates scripts only when absolutely necessary by capturing user demonstrations, rather than pre-creating scripts for all possible web sites. This partial action approach ensures resources are allocated only to web sites that actually need automated information retrieval, improving overall resource efficiency while maintaining complete coverage where needed.
Data Source
AI summary
A system and method allows users to provide portions of scripts or scripts for retrieval of information from one or more web sites of one or more businesses by demonstrating operation of the one or more web sites and identifying locations of one or more fields on each web page of the one or more web sites, and the system and method stores the scripts and uses the scripts to retrieve information from such web site or web sites for any number of users. Different portions of different scripts may be used as a single script to retrieve information from a single web site. Scripts or portions of scripts may be repaired using information from previously working scripts or portions, the web site when the script or portion worked, and the web site when the script or portion did not work.