Task scheduling method and system based on MapReduce mechanism

A task scheduling and mechanism technology, applied in the field of cloud computing, can solve the problems that cannot be solved, do not consider the isolation of malicious users, low performance efficiency, etc., to achieve the effect of reducing performance overhead

Inactive Publication Date: 2012-11-07
PEKING UNIV
View PDF2 Cites 35 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The two disadvantages of this method are: first, the performance is inefficient, because all subjobs are executed repeatedly
Second, the isolation of malicious users is not considered, and the problem of "collusion" cannot be solved, that is, both computing nodes are invaded, thus completely avoiding the integrity detection of intermediate results

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Task scheduling method and system based on MapReduce mechanism
  • Task scheduling method and system based on MapReduce mechanism
  • Task scheduling method and system based on MapReduce mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] The present invention will be described in detail below through specific embodiments and accompanying drawings.

[0032] First describe the security risks existing in the cloud computing environment built with the existing MapReduce technology, then describe the task scheduling method based on the MapReduce mechanism proposed by the present invention and the security strategy for strengthening the security features of MapReduce in the system, and describe how these security risks are resolved fixed.

[0033] 1. MapReduce security risk analysis

[0034] In order to effectively integrate computing resources, generally multiple tenants share the same cloud computing environment. The existing scheduling mechanism of MapReduce does not consider the harm that malicious users may cause to other ordinary users in a multi-tenant environment. The occurrence of hazards can be roughly divided into the following two situations:

[0035] 1) At a certain moment, the subtasks of ord...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a task scheduling method and a task scheduling system based on a MapReduce mechanism. The method comprises the following steps of: carrying out regional division on computing nodes in a MapReduce cloud computing environment, wherein a security domain, a scheduling domain and a collision domain are included; scheduling Map subtasks in a MapReduce task onto computing nodes of the security domain, and scheduling a copy of each Map subtask onto a computing node of the scheduling domain; and carrying out Hash verification on execution environments and partial intermediate results of the Map subtasks and the copies of the Map subtasks, wherein Reduce subtasks respectively utilize an execution environment verification value and an intermediate result verification value generated in a Map subtask stage to verify a self execution environment and input data. Due to the utilization of the method and the system, an operation environment of a task submitted by a common user can be effectively isolated from a malicious user, and the completeness of a task computing result can be ensured in a high-efficient way.

Description

technical field [0001] The invention belongs to the technical field of cloud computing, and specifically relates to a task scheduling method and system based on a MapReduce mechanism, which can improve the security of the MapReduce mechanism and realize the integrity of job calculation results in a more efficient manner compared with the prior art. Background technique [0002] MapReduce is a parallel computing framework suitable for cluster computing environments. Cloud service providers use MapReduce to integrate cluster computing resources and provide computing services to cloud tenants by exposing MapReduce APIs. Using the MapReduce API, cloud tenants can configure and submit computing jobs, and view the progress and results of computing jobs. [0003] For general users, MapReduce is a programming paradigm. Programs written according to the MapReduce paradigm can run in parallel on multiple nodes in the cluster; for cloud service providers, MapReduce is a programming par...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/06H04L29/08
Inventor 沈晴霓张力哲杨雅辉吴中海
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap