Webpage data acquisition method and device and electronic equipment

Through the browser extension monitoring of page changes and setting web page request hooks, efficient acquisition of asynchronous request data in dynamic web pages is achieved, the problem of low generalization of data acquisition in the existing technology is solved, and automated data recording and storage is realized.

CN119719548APending Publication Date: 2025-03-28乐麦信息技术(杭州)有限公司
0 Cites 0 Cited by

Patent Information

Application Number
CN202411717174.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-27
Publication Date
2025-03-28

AI Technical Summary

Technical Problem

In the prior art, when obtaining asynchronous request data in dynamic web pages, it is difficult to achieve high ubiquitousness, and specific scripts are required to adapt to data acquisition of different web pages.

Method used

The browser extension realizes the acquisition of web page data, uses the browser tab monitoring hook to monitor page changes in real time, enables the browser debugging mode and sets the web page request hook, receives and classifies web page requests, constructs a data storage structure, and regularly analyzes the network protocol to store the data returned by the server.

Benefits of technology

It realizes efficient acquisition of asynchronous requested data in dynamic web pages, improves the universality of data acquisition, and does not require users to write scripts to obtain data, automatically record and store data, and adapt to web page changes.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to a webpage data acquisition method and device and electronic equipment, and the method comprises the steps: carrying out the real-time monitoring of a tab change condition through a browser tab monitoring hook, and obtaining a corresponding tab ID; correspondingly starting a browser debugging mode for a browser tab corresponding to the tab ID, and setting a webpage request hook; receiving a webpage request based on the webpage request hook, classifying request messages, and constructing a corresponding data storage structure; and regularly traversing the data storage structure, analyzing the corresponding network protocol, and determining the data returned by the corresponding server and then storing the data. The webpage data are obtained by monitoring the webpage request of the user, and the request is monitored, so that the acquisition of the request cannot be influenced by how the webpage changes and how the website changes. Meanwhile, the request is initiated by the user, and the acquisition request also conforms to the mechanism of the browser, so that the data acquisition failure caused by malicious script attack is avoided.
Need to check novelty before this filing date? Find Prior Art