High-performance MapReduce realization mechanism based on dynamic migration of virtual machine

A technology of dynamic migration and virtual machine, applied in the direction of program control device, program startup/switch, software simulation/interpretation/simulation, etc., can solve the problems of unreasonable utilization of server resources and large network bandwidth occupation, and increase the overall utilization efficiency, improve security, and enhance the effectiveness of localized operations

Active Publication Date: 2014-08-27
长沙麓云信息科技有限公司
View PDF4 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0019] The present invention proposes a high-performance MapReduce implementation mechanism based on virtual machine dynamic migration, and its purpose is to overcome

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • High-performance MapReduce realization mechanism based on dynamic migration of virtual machine
  • High-performance MapReduce realization mechanism based on dynamic migration of virtual machine
  • High-performance MapReduce realization mechanism based on dynamic migration of virtual machine

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] A high-performance MapReduce implementation mechanism based on virtual machine dynamic migration, including the following steps:

[0037] Step 1: Build a distributed running Hodoop system with virtual machines on the physical server;

[0038] Step 2: The user Client submits the job to the Hodoop system described in step 1, and the Hodoop system decomposes the job into split data blocks;

[0039] Step 3: Extract sample elements from the split data block according to the rules, and then use all the extracted sample elements as a simulation sample group;

[0040] Step 4: Process the simulated sample group by the Map task to obtain the key-value pair corresponding to each split data block, and partition the data according to the key value to generate a partition;

[0041] Step 5: Each partition is processed by a Reduce task, and the Map task sourc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a high-performance MapReduce realization mechanism based on dynamic migration of a virtual machine. The high-performance MapReduce realization mechanism based on dynamic migration of the virtual machine comprises the steps of: monitoring a clustering state in real time by applying a cloud management platform of a CloudStack virtual machine, sampling split data segmentations and counting Map task sources corresponding to each key received by the virtual machine where the Reduce task exists, so as to obtain a Map task source which generates most key values; regarding the physic machine of the virtual machine where the obtained Map task resources exist as a migration target of the virtual machine where corresponding Reduce task exists; as the core, enabling the Reduce task to receive output of the Map task with the same key value locally as far as possible through dynamic migration of the virtual machine. Therefore, the network bandwidth is economized.

Description

technical field [0001] The invention relates to a high-performance MapReduce implementation mechanism based on virtual machine dynamic migration. Background technique [0002] Currently, cloud computing, as a business computing model, promotes the proposal of networked operating systems. The core technology of cloud computing is virtualization technology. As the basis of the networked operating system, the virtual machine cloud management platform is built on a cluster composed of a large number of servers and other hardware resources to provide users with an available operating system environment, and there is no connection between them. Impact, that is, the realization of IaaS (Infrastructure as a Service). [0003] Under the existing technology and operation mode, in the virtual machine cloud management platform in the cloud computing system, the virtual machine is created for the user by providing the template image of the operating system. Disk images of cloud systems...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/48G06F9/455
Inventor 唐卓江林刚曾优杨黎马稳祁玲
Owner 长沙麓云信息科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap