Unlock instant, AI-driven research and patent intelligence for your innovation.

Task scheduling method applied to distributed acquisition system

A task scheduling and collection system technology, applied in the field of big data, can solve problems such as low collection efficiency, repeated collection, and inconvenient maintenance, and achieve the effects of strong flexibility, wide application range, and strong practicability

Inactive Publication Date: 2015-09-16
INSPUR QILU SOFTWARE IND
View PDF4 Cites 28 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

There is no connection between the various collection systems, and it is very easy to have repeated collection or less collection, resulting in low collection efficiency and inconvenient maintenance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task scheduling method applied to distributed acquisition system
  • Task scheduling method applied to distributed acquisition system

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment

[0031] Specific embodiments: In actual operation, the task scheduling process is realized through the following steps.

[0032] Step 1. This method relies on the Hadoop ZooKeeper component. First, a ZooKeeper cluster must be configured in order to realize the distributed configuration of tasks and the interactive communication between services.

[0033] Step 2, deploy the collection program on each ZooKeeper server.

[0034] Step 3: Deploy the TbScheduleConsole program of TbSchedule, which is the terminal for configuring, deploying and monitoring scheduling tasks.

[0035] Step 4, configure the connection to the ZooKeeper server in the TbScheduleConsole console.

[0036] Step 4, configure the scheduling policy on the task management page, and fill in the task name, maximum number of threads, addresses of all acquisition program servers that execute the task and other information (multiple servers are separated by commas).

[0037] Step 5, configure the task. The configuratio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a task scheduling method applied to a distributed acquisition system. The task scheduling method specifically includes dynamically distributing acquisition tasks to a plurality of servers of a cluster, wherein each server is provided with an acquisition program and executes the different acquisition tasks to complete dynamic acquisition monitoring. Compared with the prior art, the task scheduling method applied to the distributed acquisition system has the advantages that batch acquisition tasks or ever-changing acquisition tasks can be distributed to applications of multiple service areas to be executed, all the tasks cannot be repeated, and quick processing without omission is achieved.

Description

technical field [0001] The invention relates to the technical field of big data, in particular to a highly practical task scheduling method applied to a distributed acquisition system. Background technique [0002] With the rapid development of the Internet, the concept of big data has attracted more and more attention. By collecting data from the Internet, enterprises can find valuable data from a large amount of data for analysis, and provide decision support through the analysis results to drive the accelerated development of enterprises. The value of data is getting more and more attention. [0003] However, there are various kinds of information on the Internet, and the scale of the Internet is becoming larger and larger. The traditional collection method requires the deployment of collection systems in a large number of servers, and the collection needs to be managed and configured server by server. There is no connection between the collection systems, and it is ver...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48
Inventor 张裕超孙海峰王传超
Owner INSPUR QILU SOFTWARE IND