Fault tolerance optimizing method of intermediate data in cloud computing environment

A cloud computing environment, intermediate data technology, applied in data exchange networks, digital transmission systems, electrical components, etc., can solve problems such as reducing the overall performance of the system, affecting the completion time of tasks, occupying network resources, etc., to solve the problem of network resource contention. use, ensure low interference, and the effect of high backup overhead

Inactive Publication Date: 2011-06-22
SHANGHAI JUNESH INFORMATION TECH CO LTD
View PDF2 Cites 29 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

While improving the fault tolerance of intermediate data, this method will occupy network resou

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Fault tolerance optimizing method of intermediate data in cloud computing environment
  • Fault tolerance optimizing method of intermediate data in cloud computing environment
  • Fault tolerance optimizing method of intermediate data in cloud computing environment

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0018] In order to make the objectives, technical solutions and advantages of the present invention more clearly expressed, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0019] The main idea of ​​the present invention is to dynamically adjust the replication of intermediate data according to the network load, task execution progress and resource usage status during task execution in the cloud computing environment, and to perform real-time control of the intermediate data replication by combining statistical network load and task execution status, To ensure that the intermediate data is replicated without interfering with the execution of the foreground task.

[0020] First classify the data to reduce replication overhead. The data generated locally will be automatically deleted after use in the next stage, while the data generated by other nodes will be automatically saved after different n...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a fault tolerance optimizing method of intermediate data in a cloud computing environment. The method comprises the following four steps of: 1. collecting a network load, task execution progress and position information; 2. judging and analyzing task input data and mainly judging a data generating position; 3. classifying working modes according to the current environmentand adjusting recourse allocation according to a control parameter; and 4. feeding back a backup condition and updating data backup information. In the invention, the frequency of node failure in thecloud computing environment is taken into consideration at first, the intermediate data is replicated according to the task finishing time requirement and the resource use condition, and real-time monitoring and flexible management are carried out on the network load and task execution progress so as to ensure that the replication of the intermediate data is realized for the system under the condition that performance is free from influence. The fault tolerance optimizing method provided by the invention has wide practical value and application prospect in the field of cloud computing data management.

Description

technical field [0001] The present invention designs a fault-tolerant optimization method for intermediate data in a cloud computing environment, specifically designs a data management method in which intermediate data in a cloud computing environment adopts selective storage, performs background copying and cost comparison, etc., reduces storage copying costs, and improves the system Performance belongs to the technical field of computer storage. Background technique [0002] With the popularity of cloud computing, the importance of cloud computing programming model has become increasingly prominent, among which the data flow programming model is a dominant programming model. Dataflow programming models are increasingly popular for large-scale data processing, such as MapReduce, Dryad, and Pig. Dataflow programs are computations that include multiple stages and the means of communication that connect the stages together. The intermediate data is the data generated in each...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L12/24H04L29/08
Inventor 祝明发胡声秋肖利民阮利毛宏
Owner SHANGHAI JUNESH INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap