Task optimal scheduling method

A scheduling method and task optimization technology, applied in the computer field, can solve problems such as the scheduling scheme is not the optimal solution and the resource utilization rate is low.

Active Publication Date: 2016-12-21
UNIV OF SCI & TECH BEIJING
View PDF6 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The technical problem to be solved by the present invention is to provide a task optimization scheduling method to solve the low utilization rate of resources in heterogeneous cluste

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task optimal scheduling method
  • Task optimal scheduling method
  • Task optimal scheduling method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0052] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will describe in detail with reference to the drawings and specific embodiments.

[0053] The present invention aims at the problem that the resource utilization rate in the existing heterogeneous cluster is low, and the scheduling scheme is not dynamically adjusted according to the current data locality constraints, resulting in the scheduling scheme not being the optimal solution, and provides a task optimization scheduling method.

[0054] see figure 1 As shown, the task optimization scheduling method provided by the embodiment of the present invention includes:

[0055] S1. Obtain the correlation between any two data blocks in a data center (DC, Data Center), and construct a correlation matrix between any two data blocks in the DC, wherein the correlation between any two data blocks The reliability is determined by the number of t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a task optimal scheduling method and can improve the task execution performance. The method comprises the following steps: S1, constructing a data dependency matrix between any data blocks in a DC (Data Center); S2, converting the data dependency matrixes into data clustering matrixes; S3, carrying out submatrix segmentation on the data clustering matrixes according to maximum available storage capacities of machine frames and of nodes in the machine frames, and placing the data blocks corresponding to the data clustering matrixes onto the nodes in the machine frames according to a segmentation result to obtain a current data placement scheme; S4, under the constraints of the current data placement scheme, with data migration cost minimization as a goal, determining a current optimal task scheduling scheme; and S5, when a data locality degree and a data migration cost in task execution process meet predetermined conditions, executing steps S1 to S4 repeatedly; or otherwise, outputting the current data placement scheme and the current task scheduling scheme as optimal solutions for task scheduling. The task optimal scheduling method provided by the invention is applied to the technical field of computers.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a task optimization scheduling method. Background technique [0002] In recent years, big data processing frameworks such as MapReduce have played an increasing role in processing data-intensive applications. It cannot be ignored that in practical applications, there is a correlation between the input data of some businesses. This correlation can be expressed as several data blocks may be processed by the same task, or as the input data of the next stage contains Intermediate results from the previous stage. Taking user profile information in social networks as an example, users who are in the same social circle or have common attributes in a certain aspect have correlations between their data information. When analyzing user data, if such user information is merged / aggregated, the size of task output results can be effectively reduced, thereby reducing data migration during ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/50
CPCG06F9/5088
Inventor 阳小龙王玢孙奇福隆克平
Owner UNIV OF SCI & TECH BEIJING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap