Method and system for acquiring distributed data

A distributed data and acquisition method technology, applied in the field of distributed data acquisition methods and systems, can solve problems such as failure to acquire data, difficulty, and small amount of acquired data, so as to easily find errors, improve utilization, and improve data accuracy. Effect

Inactive Publication Date: 2018-03-27
台州市吉吉知识产权运营有限公司
View PDF4 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The current distributed data acquisition method is generally based on swoole to obtain distributed data; however, it is necessary to introduce a third-party plug-in swoole, which is cumbersome to use, compatibility issues need to be considered, and it is inconvenient to debug the program and improve the error program.
[0005] At present, using PHP as a Kafka consumer to obtain data from Kafka is relatively simple, using a single process method, often failing to obtain data, making it difficult to find errors, slow in obtaining data, and small in amount of data obtained And other issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for acquiring distributed data
  • Method and system for acquiring distributed data
  • Method and system for acquiring distributed data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation manners of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other accompanying drawings based on these drawings and obtain other implementations.

[0035] In order to make the drawing concise, each drawing only schematically shows the parts related to the present invention, and they do not represent the actual structure of the product. In addition, to make the drawings concise and easy to understand, in some drawings, only one of the components having the same structure or function is schematically shown, or only one of them is marked. Herein, "a" not only means "only one", but also means "more than one".

[0036] like figu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and system for acquiring distributed data. The method includes the steps that 1, a daemon process is created; 2, the daemon process monitors whether or not message pushof original data exists; 3, if the message push of the original data is not monitored, the daemon process continues keeping a monitoring state; 4, if the message push of the original data is monitored, a plurality of sub-processes are created according to the theme type of the original data, and the daemon process allocates the original data to the corresponding sub-processes; 5, the sub-processes acquire the distributed data from the original data. By means of the method and system for acquiring the distributed data, the distributed data is acquired more conveniently and efficiently, and thesuccess rate is higher.

Description

technical field [0001] The invention relates to the field of data collection, in particular to a distributed data acquisition method and system. Background technique [0002] There are many types of distributed data. In fact, the data in the Kafka cluster is one of them. The full name of kafka is Apache Kafka. It is a distributed messaging system originally developed and open sourced by Linkedin. It is widely used because of its distributed and high throughput. It can handle all the action flow data in the website, such action (eg search, browse) is a key factor of many social functions in the modern web. [0003] Several concepts are involved in Kafka, such as producers, consumers, brokers, topics, etc. The producer is the party that produces data, English is producer; the consumer is the party that uses data, English consumer; Broker is the node of the kafka cluster, each kafka cluster is composed of multiple brokers, these brokers are interrelated and jointly provide S...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/44G06F9/445G06F11/30G06F11/34
CPCG06F9/4451G06F11/302G06F11/3404
Inventor 陈云龙
Owner 台州市吉吉知识产权运营有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap