Web Data Extraction System with Visual Event Configuration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting web data are complex and require specific technical knowledge, making them inaccessible to general users and inefficient for intuitive data extraction from various web sources.
Innovation Solution
A system that allows users with basic knowledge to selectively import web data by arbitrarily setting action design, comprising a loading unit, action design unit, executing unit, and importing unit, which replicates the target web site, enables users to plan and perform extraction tasks visually through event creation and queuing signals to external BOTs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If existing web data extraction methods are used, then data extraction capability is achieved, but system complexity and technical knowledge requirements increase
Solution Approach 1:
The patent introduces an event-driven intermediary layer between the user interface and the web data extraction engine. Users define extraction scenarios through simple event configurations (click, hover, input events) rather than directly programming complex extraction logic. This event mediator translates user-friendly event definitions into detailed extraction instructions, reducing the technical knowledge barrier while maintaining extraction capability.
Solution Approach 2:
The system segments the web data extraction process into discrete, independently configurable events. Each event represents a specific user interaction (click, hover, input) that triggers a specific extraction action. This segmentation allows users to build complex extraction scenarios by combining simple, well-defined event blocks, reducing overall system complexity through modular design.
2Adaptability or versatility
If existing web data extraction methods are used, then data extraction capability is achieved, but ease of use for general users deteriorates
Solution Approach 1:
The system enables general users to perform web data extraction through self-service event configuration. Users can define their own extraction scenarios by selecting from predefined event types and configuring parameters through an intuitive interface, without requiring assistance from experts or knowledge of complex extraction techniques. The system provides built-in guidance and automation that empowers users to independently complete extraction tasks.
3Ease of operation
If visualized flow chart method is used, then user-friendly interface is improved, but system architecture complexity increases
Solution Approach 1:
The system uses visualized flow charts as a simplified copy or representation of the underlying event-driven architecture. The flow chart interface presents a user-friendly visual model that mirrors the actual event sequence and data flow, allowing users to design and understand extraction scenarios without directly interacting with the complex system architecture. This visual copy bridges the gap between simple user interaction and complex system operation.
Data Source
AI summary
The present invention discloses a loading unit which loads the target web site after receiving a URL of a target web site; an action design unit which makes the user arbitrarily plan and arbitrarily set pre-determined work to extract target data from the target web site; an executing unit which consistently arranges the pre-determined work arbitrarily planned through the action design unit, and performs a designated event; an importing unit which extracts target data through the event work and presents them to a user.