Method and system for task distribution

A task allocation and task technology, applied in the field of data processing, can solve problems such as waste of resources, low processing efficiency, task starvation, etc., and achieve the effect of improving utilization rate, improving processing efficiency, and reducing waste of resources

Active Publication Date: 2013-09-11
阿里巴巴(中国)网络技术有限公司
View PDF4 Cites 29 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The present application provides a task allocation method and system to solve the problem of relatively long processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for task distribution
  • Method and system for task distribution
  • Method and system for task distribution

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] In order to make the above objects, features and advantages of the present application more obvious and comprehensible, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0026] In a distributed server cluster (the embodiment of this application takes MySQL server as an example, but is not limited to MySQL server cluster), the statistical result data is calculated on the distributed computing platform, and the calculated data is imported in batches Each MySQL server, and then perform subsequent processing. Wherein, the calculated data can be imported in the form of a text file, and the import process is realized by using a priority work queue. For disaster recovery considerations, these MySQL servers are distributed in different computer rooms. If the calculated data is used as a task, each worker processing the task is also distributed in different computer rooms.

[00...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and a system for task distribution so as to solve the problems in the task distribution method in the prior art. The method comprises the steps as follows: enabling a task to enter a work queue, and distributing a target server of the task; when the task is deleted from the work queue, if an idle worker is existent on the target server, seizing and processing the task by the idle worker; and if the idle worker is inexistent on the target server, searching the idle worker closest to the target server of the task in all servers, and seizing and processing the task by the idle worker, wherein the worker is a task processing process or a task processing thread. The task is processed by the idle worker after the task is deleted from the task queue, so that the processing time is effectively shortened and the processing efficiency is improved; and meanwhile, the idle worker is selected according to the distance, so that the resource waste can be reduced.

Description

technical field [0001] The present application relates to data processing technology, and in particular to a task allocation method and system. Background technique [0002] In a distributed server cluster (such as a MySQL server cluster), after statistical result data is usually calculated on a distributed computing platform (such as Hadoop), the calculated data will be imported in batches into each MySQL server, and then Then carry out subsequent processing. Wherein, the import process is implemented by using a priority work queue. For disaster recovery considerations, these MySQL servers are distributed in different computer rooms. If the calculated data is used as a task, each worker processing the task is also distributed in different computer rooms. Wherein, the worker is a process or thread for processing tasks. [0003] Usually, tasks are assigned in a pre-assigned manner, that is, a worker is assigned to a task when it enters the queue, which means that the worke...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/46G06F9/50
Inventor 张轩丞
Owner 阿里巴巴(中国)网络技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap