Unlock instant, AI-driven research and patent intelligence for your innovation.

Web page information acquisition system

A web page information and acquisition system technology, applied in the field of computer networks, can solve the problem of not being able to meet the needs of customized web page information extraction

Active Publication Date: 2016-04-13
BEIJING QIHOO TECH CO LTD
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This type of crawler will not analyze the information of the webpage, and cannot meet the needs of customizing the extraction of webpage information

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Web page information acquisition system
  • Web page information acquisition system
  • Web page information acquisition system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0017] figure 1 A flowchart of a method 100 for obtaining webpage information according to an embodiment of the present invention is shown. Such as figure 1 As shown, the method 100 begins with step S101, which is a crawling step, specifically crawling a web page from a web server. The crawler system can use the following three methods to crawl web pages from the site server: 1) Download web pages directly from the site server. T...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a system for acquiring webpage information. The system comprises a device for acquiring webpage information and a station server, wherein the device for acquiring webpage information comprises a webpage grabber which is suitable for grabbing webpages from the station server, a webpage information resolver which is suitable for a preset webpage extraction rule and suitable for extracting specified webpage information from specified positions of webpages according to a preset webpage extraction rule, and an action processor which is suitable for structurally storing the specified webpage information. According to the system for acquiring webpage information provided by the invention, after webpages are grabbed from the station server, information of entire webpages is not stored directly, and specified webpage information is extracted from specified positions of the webpages according to the webpage extraction rule and is structurally stored. The webpage extraction rule can be customized according to the requirements of users, and information of webpages is resolved, so that the requirement of customized extraction of webpage information is met.

Description

technical field [0001] The invention relates to the technical field of computer networks, in particular to a web page information acquisition system. Background technique [0002] Web crawler (also known as web spider, web robot, and in some communities, more often referred to as web chaser) is a program or script that automatically obtains web content. It is an important part of search engines. Engine optimization is largely optimized for web crawlers. [0003] Web crawlers are generally divided into traditional crawlers and focused crawlers. Traditional crawlers start with one or several initial web page URLs (Uniform / UniversalResourceLocator, Uniform Resource Locator), and obtain the initial web page URL; in the process of crawling web pages, they continuously extract new web page URLs from the current page and put them into the queue , until a certain stopping condition of the system is met. The work flow of the focused crawler is relatively complicated, and it is nec...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 徐锐波路轶
Owner BEIJING QIHOO TECH CO LTD