Universal method for capturing webpage contents of any webpage

Abstract

The invention discloses a universal method for capturing webpage contents of any webpage, which belongs to the technical field of networks. The technical scheme is a universal method for capturing webpage contents of any webpage. The method comprises the following steps: inputting the captured websites and conditional expressions, creating a subpage and transmitting the websites and the conditional expressions to the subpage by a user; requesting a server, acquiring all contents of webpages of the websites and embedding a segment of javascript program into the webpage contents by the subpage; converting the conditional expressions into an array variable, traversing the array and finding out all nodes according with the conditions by the javascript program; and acquiring inner HTML(Hypertext Markup Language) or outer HTML(Hypertext Markup Language) attribute value of all nodes and further acquiring corresponding webpage contents by the javascript program. The method can ensure that the user can capture any content of the webpages by only simply modifying the conditional expressions without writing a code for analyzing the webpage contents aiming at each web page.

Description

technical field [0001] The invention belongs to the field of network technology, in particular to a general method for grabbing webpage content that can be used for any webpage. Background technique [0002] In the Internet age, abundant Internet resources have greatly facilitated people's information life. However, with the explosive increase of the amount of information, as an ordinary netizen, if he can easily and conveniently grab the valuable information content from any other webpage on the Internet and use it on his own webpage, it will become a A headache for netizens. Because the traditional crawling method has high technical barriers, in order to capture a certain piece of information content of a certain web page, it is often necessary to perform a complex analysis of the data content of the web page, and finally extract the information content you need. And once you change another webpage to grab its webpage content, you have to redesign the analysis code of th...

AI Technical Summary

Problems solved by technology

However, with the explosive increase of the amount of information, as an ordinary netizen, if he can easily and conveniently grab the valuable information content from any other webpage on the Internet and use it on his own webpage, it will become a Netizens' headaches

Because the traditional crawling method has high technical barriers, in order to capture a certain piece of information content of a certain web page, it is often necessary to perform a complex analysis of the data content of the web page, and finally extract the information content you need. And once you change another webpage to grab its webpage content, you have to redesign the analysis code of the program. This process involves a lot of work duplication and inefficiency, because all the analysis work requires you to design the analysis code yourself, and It does not use the system's native functions for analysis, so it is often easy to analyze errors, and it is difficult for ordinary netizens to perform such complex operations

Images