Network information batch acquisition method of combined text and picture information

Abstract

The invention provides a network information batch acquisition method of combined text and picture information. Through a series of configuration, according to the method, target network information can be acquired, replication removal of the target network information can be achieved, and the target network information can be stored into a database and can be sent to a place designated by a client according to a format designated by a client. The method includes the steps that websites in need of information acquisition and specific URLs and the page number of information list pages are determined; according to the URLs of the list pages, and a common part is found out and stored in list configuration information; when information acquisition is conducted, a system reads URL common part information in the list configuration information, serial number information of all the list pages is obtained according to the total number of the list pages, and therefore the URLs of all the list pages to be acquired of a target network are combined; detailed page content capturing is conducted according to detailed page link addresses stored in a linking library to be captured; processing of pictures in detailed page contents is conducted; after the detailed page contents are captured, content data are led out to a designated interface.

Description

technical field [0001] The invention is applied in the technical field of the Internet, and relates to a batch collection method of network information combining text and picture information. technical background [0002] With the rapid development of the Internet, a large amount of various information has accumulated on the Internet, such as news information, potential customer information, price information of competitive products, real-time financial information, statistical reports, industry analysis reports, supply and demand information, etc. For enterprises, the analysis of these information combined with the internal business data of the enterprise has a great auxiliary effect on the business decision-making of the enterprise. The content of the enterprise website is also very helpful to improve the experience of visitors. [0003] Now there are many tools that can realize the collection of webpage content, but the collection method of text information is the main m...

AI Technical Summary

Problems solved by technology

However, this method does not deal with abnormal situations in the collection process, cannot correctly identify redundant codes in information pages, and does not process pictures in web pages

Images