Data acquisition system of login type website

A data acquisition system and data acquisition technology, applied in the direction of network data retrieval, website content management, electronic digital data processing, etc., can solve the problems that cannot meet the timeliness and integrity of user information, achieve rapid horizontal expansion, reduce Labor cost and error rate, and the effect of reducing manual intervention

Inactive Publication Date: 2016-05-04
HYLANDA INFORMATION TECH
View PDF3 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The current basic collection methods for such situations can no longer meet the user's requirements for timeliness, integrity and other aspects of information

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data acquisition system of login type website
  • Data acquisition system of login type website

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0017] The present invention will be described in detail below by means of the accompanying drawings and specific embodiments.

[0018] Such as figure 1 and figure 2 As shown, the data collection system of the login type website of the present invention includes the following modules: a collection task management module, a collection account management module, a collection policy management module, a collection worker module, a collection result processing module and a system status management module; The management module distributes download tasks, abstracts various data collection tasks into a unified collection task object, generates a collection task queue, and hands the subsequent collection workers to complete the data collection work; the collection account management module manages the collection worker module in the work. The website account used; the collection policy management module completes the work of collection resource allocation and collection strategy ch...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data acquisition system of a login type website. The data acquisition system of the login type website comprises an acquisition task management module, an acquisition account management module, an acquisition strategy management module, an acquisition worker module, an acquisition result processing module and a system state management module; and various types of acquisition tasks can be unified into a standard acquisition task object to meet almost all description requirements on data acquisition work. An acquisition account management platform based on an HTTP (Hyper Text Transport Protocol) carries out unified management on accounts which need to be used for each class of website to log, uniformly distributes accounts according to requirements, and automatically analyzes, processes and records information including the use situations of the accounts, the current state and the like according to result information. A task-driven-based acquisition work strategy mechanism can dynamically regulate an acquisition resource ratio, change an acquisition strategy and realize the quick and horizontal expansion of system resources according to the requirements of the system. A highly automated management mechanism can greatly lower manual intervention work and lower labor cost and an error rate.

Description

technical field [0001] The invention relates to the technical field of Internet information collection, in particular to a data collection system for a login type website. Background technique [0002] With the development of the Internet industry, many websites only provide "limited data display" for "anonymous" visitors, and they need to obtain their "registered membership" and log in to their websites before the other party can provide more detailed data. The current basic collection methods for such situations can no longer meet the user's requirements for timeliness, integrity and other aspects of information. Contents of the invention [0003] The technical problem to be solved by the present invention is to provide a data acquisition system for a login type website. [0004] The technical scheme that the present invention takes for solving the technical problem existing in known technology is: [0005] The data collection system of the login type website of the pr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/958G06F16/955
Inventor 郭猛周祖胜
Owner HYLANDA INFORMATION TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap