Web Page Service Packaging via Segmentation and Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing service packaging systems for web pages are inefficient, as they require significant manual effort and technical expertise to extract and package data, limiting developer productivity and data acquisition efficiency.
Innovation Solution
A service packaging method based on web page segmentation and search algorithms, which includes dynamic and static packaging stages, allowing users to parse and tag web pages with minimal input, generating structured data and crawler rules to create a service that can be called with a few clicks, using tools like Selenium, BeautifulSoup, and Pyquery for crawling and image processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual data extraction and packaging is performed, then data acquisition accuracy can be maintained, but the time consumption and operational complexity increase significantly
Solution Approach 1:
The system enables self-service automated packaging by allowing users to define extraction rules once, after which the system automatically parses web pages, extracts data, and generates services without requiring manual intervention for each packaging task
Solution Approach 2:
The system performs preliminary analysis of web page structures to pre-identify extraction patterns and data fields, so that when packaging is needed, the extraction process can proceed efficiently using pre-established rules and templates
2Adaptability or versatility
If complex packaging processes are used to handle various web page formats, then adaptability improves, but the system complexity and difficulty of operation increase
Solution Approach 1:
The system segments web pages into standardized data blocks using automated parsing, dividing complex page structures into manageable, uniformly formatted units that can be easily processed and packaged regardless of the original page complexity
Solution Approach 2:
The system creates a universal packaging framework that can handle multiple web page formats and structures through a single standardized interface, allowing one system to serve multiple packaging needs without requiring format-specific complex procedures
3Productivity
If automated parsing is implemented, then packaging efficiency increases, but the initial system setup and rule definition complexity increases
Solution Approach 1:
The system uses template copying where standard packaging templates and extraction rules can be replicated and adapted for similar web page structures, reducing the need to create complex configurations from scratch for each new packaging task
Data Source
AI summary
The present invention provides a service packaging method based on web page segmentation and search algorithm, comprising the following steps: a service extraction stage, comprising dynamic packaging and/or static packaging; for dynamic packaging, parsing a dynamic web page, tagging forms that possibly exist in parsed dynamic form information, and tagging and defining, by a user, desired forms among the forms that possibly exist; for static packaging, parsing a static web page, blocking and tagging parsed static forms, and selecting and defining, by the user, desired blocks, and filling in a name, description information and an extraction rule of a service; and a service calling stage, comprising inputting, by the user, related information for calling a service, and generating, by a back end system, a corresponding service according to the received related information for calling the service and according to the extraction rule, and returning the corresponding service to a front end. The present invention greatly increases the efficiency of acquiring data by a user.