Data intensive computing cost optimization method based on MapReduce mechanism

A data-intensive, cost-optimized technology, applied in electrical digital data processing, special data processing applications, instruments, etc., to solve problems such as computing time and cost waste

Inactive Publication Date: 2014-02-19
UNIV OF SCI & TECH BEIJING
View PDF2 Cites 23 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to provide a data-intensive computing cost optimization method based on the MapReduce mechanism, which overcomes the above-mentioned shortages of file contention and limited resources of hot and busy data, and solves the problems caused by the dynamic access of...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data intensive computing cost optimization method based on MapReduce mechanism
  • Data intensive computing cost optimization method based on MapReduce mechanism
  • Data intensive computing cost optimization method based on MapReduce mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0033] It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0034] A data-intensive computing cost optimization method based on the MapReduce mechanism described in the embodiment of the present invention dynamically changes the number of copies of the Map subtask according to the popularity of the data in the load; Increase the number of copies to reduce file snatching and network bandwidth resource competition, so as to optimize the calculation cost and calculation time; for cold data with less access times and less frequent access, reduce the number of copies to reduce the creation of copies and storage cost to optimize its computing cost and computing time.

[0035] The specific process is:

[0036...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a data intensive computing cost optimization method based on a MapReduce mechanism. In the state that the popularity level and access situations of data change, execution time and execution cost for a MapReduce task are optimized. The method comprises the steps that a cost computing model based on the MapReduce mechanism in a cloud computing environment is provided; on the basis of the model, the number of copies of Map subtasks is dynamically changed according to the popularity level of the data in a load; as for hot data which have the large number of access times and are accessed frequently, file contention and bandwidth resource competition of networks are reduced by increasing the number of copies of the hot data, and therefore the computing cost and the computing time for the hot data are optimal; as for cold data having the small number of access times and are accessed seldom, by decreasing the number of copies of the cold data, creation and storage cost for the copies is reduced, and therefore the computing cost and the computing time for the cold data are optimal. By means of the data intensive computing cost optimization method, cloud computing users can effectively reduce resource using cost, and reliability and completeness of operation computed results can be guaranteed more efficiently.

Description

technical field [0001] The present invention relates to a cost-based optimization method, more specifically to a data-intensive computing cost optimization method based on a MapReduce mechanism. This method is suitable for the optimization of computing cost and computing time for cloud computing users when the popularity of data in the load and the access situation change dynamically. Background technique [0002] With the development of information technology, more and more industrial and academic organizations are facing the challenge of dealing with growing large-scale data, such as file analysis processing, scientific simulation and other applications. As the most mainstream programming model for processing large-scale data and data-intensive applications, MapReduce effectively decomposes large-scale input data sets into fixed-size data blocks, which are distributed and stored in different nodes in the cluster, and distributed parallel computing model , which greatly si...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/184
Inventor 杨扬孙莉莉米振强
Owner UNIV OF SCI & TECH BEIJING
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap