Method for collecting internet data in one step

An Internet and data technology, applied in the field of information retrieval, can solve problems such as rising data collection costs, and achieve the effect of short collection time, avoiding data leakage, and omitting development.
CN113407803AInactive Publication Date: 2021-09-17浪潮卓数大数据产业发展有限公司
0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
浪潮卓数大数据产业发展有限公司
Publication Date
2021-09-17
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 1
    Figure 1
Patent Text Reader

Abstract

The invention particularly relates to a method for collecting internet data in one step. The method for collecting internet data in one step comprises the following steps: firstly, collecting a request parameter, and carrying out custom setting on the request parameter for website retrieval; then sending an access request to the website through the user-defined request parameters, and obtaining the response information conforming to the user-defined data display parameters; carrying out one-time collection on website data to obtain total data; and finally, performing structured processing on the obtained retrieval result information, and storing the retrieval result information in a database to achieve the purpose of data collection. According to the method for collecting the internet data in one step, the request parameters related to data display are specified, so that the total target data of the website is accessed and collected at one time, the collection time is shorter, and collection resource nodes are not occupied, so that data collection omission is avoided, the data can be quickly and efficiently collected, development of a crawler program is simplified or omitted. The convenience is provided for subsequent data storage and data analysis.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of information retrieval, in particular to a method for collecting Internet data in one step. Background technique

[0002] Modern society is a rapidly developing society. With the rapid development and popularization of computer and information technology, the scale of industrial application systems is expanding rapidly, and the data generated by industrial applications is growing explosively. People are increasingly aware of the importance of data. This concept has attracted widespread attention from practitioners and users in various industries. The mining and application of massive data heralds the arrival of a new wave of productivity growth and consumer surplus. With the development of government information disclosure and enterprise digitization, a large amount of valuable data can be obtained through the collection of Internet data.

[0003] Information Retrieval (Information Retrieval), usually re...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More