Collection method and device based on distributed message system

A technology of a message system and a collection method, applied in the field of data processing, can solve problems such as low efficiency, and achieve the effect of improving collection efficiency

Inactive Publication Date: 2016-05-25
INSPUR QILU SOFTWARE IND
View PDF3 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] It can be seen that the traditional collection method requires manual assignment of collection tasks, which is inefficient

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Collection method and device based on distributed message system
  • Collection method and device based on distributed message system
  • Collection method and device based on distributed message system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Apparently, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0050] An embodiment of the present invention provides a collection method based on a distributed message system, which is applied to a distributed message system, wherein the distributed message system is connected with at least one collection node and a management terminal, such as figure 1 As shown, the method may include the following steps:

[0051] Step 101:: configure task theme;

[0052] Step 102: establishing an automatic assignment rule, the au...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a collection method and device based on a distributed message system. The method includes the steps that an automatic allocation rule is established; a target collection task sent from a management end to a task topic is received; the target collection task is sent to one or more collection nodes according to the automatic allocation rule so that the collection nodes can execute data collection according to the target collection task; sub tasks returned to the task topic by the collection nodes when it is determined that the collected data contain the sub tasks are received, and the sub tasks are taken as the target collection task; the receiving step and the sending step are executed repeatedly till no sub task returned by the collection nodes is received in the set time period is received. By means of the collection method and device, automatic allocation and collection of the task can be achieved, and collection efficiency is improved.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a collection method and device based on a distributed message system. Background technique [0002] With the rapid development of the Internet, the concept of big data has attracted more and more attention. Enterprises can collect data from the Internet through web crawlers, and further process the data to find more valuable data analysis results, and provide decision support through the analysis results to drive the accelerated development of enterprises. [0003] The traditional data collection method crawls the list URL of the collection task, and manually assigns the list URL to the collection node for collection. [0004] It can be seen that the traditional collection method requires manual assignment of collection tasks, which is inefficient. Contents of the invention [0005] The invention provides a collection method and device based on a distributed message s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/955G06F16/1815G06F16/951
Inventor 张裕超王传超孙海峰
Owner INSPUR QILU SOFTWARE IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap