Data collection system and method

A data acquisition system and data acquisition technology, applied in the field of data processing, can solve the problems of difficult to deal with diversified enterprise data, multiple storage methods and concurrent storage methods, etc., to achieve the effects of achieving diversity, controlling load balance, and speeding up processing cycles.

Active Publication Date: 2014-09-24
YONYOU NETWORK TECH
View PDF7 Cites 30 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most of the current data acquisition systems only aim at a specific data source and store the specific data source in a specific file system or data warehouse, which makes it difficult to deal with the diverse enterprise data in practical applications and the following The concurrent s

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data collection system and method
  • Data collection system and method
  • Data collection system and method

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0032] In order to be able to understand the above objectives, features and advantages of the present invention more clearly, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the application and the features in the embodiments can be combined with each other if there is no conflict.

[0033] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described here. Therefore, the protection scope of the present invention is not limited to the specific details disclosed below. Limitations of the embodiment.

[0034] figure 1 A block diagram of a data acquisition system according to an embodiment of the present invention is shown.

[0035] Such as figure 1 As shown, the data collection system 100 according to t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data collection system based on a Zookeeper cluster. The system comprises a message queue establishing unit and the Zookeeper cluster. The message queue establishing unit is used for establishing different message queues according to different data sources. The Zookeeper cluster is used for monitoring the states of a plurality of collectors, and enabling any message datum in the message queues to be collected by only one collector, wherein each message queue corresponds to a persistence node of the Zookeeper cluster. Each collector obtains message data under the coordination of the Zookeeper cluster, and corresponds to the temporary node of the persistence node. Correspondingly, the invention further provides a data collection method based on the Zookeeper cluster. According to the technical scheme, different collection modules can be arranged according to different data sources, different storage strategies are set according to different storage requirements, and diversity of data collection and storage is achieved.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a data acquisition system based on a Zookeeper cluster and a data acquisition method based on a Zookeeper cluster. Background technique [0002] With the advent of the big data era, the amount of enterprise data is increasing dramatically, and the data generated by various enterprise applications needs to be collected and stored on the big data platform in a real-time, efficient and secure manner. [0003] The current data collection system has a variety of solutions to choose from, such as distributed data collection tools Apache Flume, Facebook Scribe, etc. However, most of the current data acquisition systems only aim at a specific data source and store the specific data source in a specific file system or data warehouse, which makes it difficult to deal with the diverse enterprise data in practical applications and the following The concurrent storage method based on...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08G06F17/30G06F9/46
Inventor 费英林
Owner YONYOU NETWORK TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap