Script Matching via Page and Code Embeddings for Changing Web Pages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scripts struggle to adapt to changes in web page structure, requiring substantial programming changes or new script development, which is time-consuming and costly, and machine learning techniques fail to identify suitable scripts for new web pages.
Innovation Solution
A machine learning model ensemble is trained using twin tower training to embed web page and script logic into vectors, enabling automatic identification and modification of scripts to extract data from evolving websites.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If web page structure changes, then data extraction functionality is lost, but rewriting scripts is time-consuming and costly
Solution Approach 1:
The system enables self-service by automatically matching changed web pages to suitable existing scripts through machine learning models. The comparison model and feature embedding models work autonomously to identify and recommend appropriate scripts without human intervention, allowing the system to adapt to web page changes automatically.
Solution Approach 2:
The patent replaces manual script writing and modification (mechanical programming work) with an automated machine learning-based matching system. The models compare web page features with script features to automatically identify suitable scripts, substituting the manual mechanical process of script development with an intelligent automated system.
2Adaptability or versatility
If web page structure changes substantially, then existing scripts become non-functional, but creating new scripts is difficult and costly
Solution Approach 1:
The patent introduces feature embeddings as an intermediary representation layer between web pages and scripts. The page feature embedding model and script feature embedding model convert raw web page and script data into comparable feature vectors, enabling the comparison model to bridge the gap between changed web pages and existing scripts without requiring direct manual analysis or modification.
Solution Approach 2:
The system changes the parameters of comparison by using machine learning models to extract and compare relevant features rather than performing exact structural matching. The feature embedding models transform web pages and scripts into numerical representations that capture essential characteristics, allowing flexible adaptation to structural changes while maintaining functional compatibility.
3Ease of operation
If manual script adjustment is attempted, then programming expertise is required, but such expertise is not readily available
Solution Approach 1:
The system performs self-service by automatically identifying suitable scripts for changed web pages without requiring human programming expertise. The machine learning models autonomously analyze web page features, compare them with script features, and recommend appropriate scripts, eliminating the need for manual intervention by skilled programmers.
Solution Approach 2:
The patent replaces manual programming work (mechanical script writing) with an automated intelligent system. The comparison model and feature embedding models perform the analytical and matching tasks that previously required human programming expertise, substituting human cognitive work with automated machine learning processes.
Data Source
AI summary
A method including extracting a number of page features from a web page. The number of page features represent an executable logic of the web page. The method also includes embedding, by a page feature embedding model, the number of page features to generate a page vector data structure. The method also includes comparing, by a comparison model, the page vector data structure and a number of script vector data structures to identify a selected script. Each of the number of script vector data structures is generated by a script feature embedding model processing computer executable program code of a corresponding script for performing a computer function on a web page. The method also includes presenting the selected script.