Unlock instant, AI-driven research and patent intelligence for your innovation.

Computation task adjusting method and device

A technology for calculating tasks and adjusting methods, which is applied in the field of distributed computing and can solve problems such as large time differences

Active Publication Date: 2017-11-17
BEIJING QIYI CENTURY SCI & TECH CO LTD
View PDF4 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the embodiments of the present invention is to provide a calculation task adjustment method and device to solve the problem that the time required for each node to perform calculation processing on the corresponding calculation task varies greatly

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Computation task adjusting method and device
  • Computation task adjusting method and device
  • Computation task adjusting method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0091] The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.

[0092] In order to solve the problem that the time required by each node to perform calculation and processing on corresponding calculation tasks varies greatly, embodiments of the present invention provide a calculation task adjustment method and device.

[0093] A computing task adjustment method provided by the present invention is firstly introduced below.

[0094] MapReduce is a distributed computing framework that includes two computing phases, Map and Reduce.

[0095] Among them, a calculation task adjustment method provided by the present invention is applied to the resource manager, and the resource manager records the corresponding relationship between each Mapper and each assigned node, a Mapper is a calculation task, and each node performs a corresponding task Computing and processing; In a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention provides a computation task adjusting method and device. The method comprises the following steps: determining a target node corresponding to a target computation task; acquiring target parameter information corresponding to the target node; computing total target time consumption of the target node performing computation on the target computation task according to the target parameter information; and adjusting the target computation task according to the total target time consumption and the total time consumption corresponding to other computation tasks excepting the target computation task and based on the time consumption balance principle. According to the method, the influence factor in computation task computation processing of each node is considered, and each computation task is adjusted, so that the difference of time for each node computing the corresponding computation task is reduced, and as a result, the efficiency of MapReduce computing the whole file can be increased.

Description

technical field [0001] The invention relates to the technical field of distributed computing, in particular to a computing task adjustment method and device. Background technique [0002] MapReduce is a distributed computing framework that includes two computing phases, Map and Reduce, where Map is mapping and Reduce is protocol. In the prior art, the input file of MapReduce is divided into multiple file blocks of equal size, each file block is called a split, each split corresponds to a corresponding Mapper, each computing task is a computing task, and Each Mapper corresponds to a corresponding node, and the node is used to calculate and process the corresponding Mapper, that is, the node calculates and processes the file content of the split corresponding to the Mapper. The Reducer reads the output results of the nodes corresponding to the Mapper, and the nodes corresponding to the Reducer perform reduction and combined calculations according to the received calculation r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48
CPCG06F9/4881
Inventor 梁建煌
Owner BEIJING QIYI CENTURY SCI & TECH CO LTD