A method and system for elastic resource expansion based on machine learning

A technology of machine learning and extension method, which is applied in the field of cloud computing, can solve the problems of low estimation accuracy and difficulty in elastic resource management, and achieve high accuracy, solve the effects of computing speed reduction and elastic resource management

Active Publication Date: 2021-02-05
HUAZHONG UNIV OF SCI & TECH
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for expanding elastic resources based on machine learning, thereby solving the technical problems of difficult elastic resource management and low estimation accuracy in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and system for elastic resource expansion based on machine learning
  • A method and system for elastic resource expansion based on machine learning
  • A method and system for elastic resource expansion based on machine learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0099] In order to verify the feasibility and effectiveness of the method of the present invention, the method of the present invention is verified in a real environment. Experimental preparations include: setting up a cluster with a maximum number of 40 virtual machines on the Alibaba Cloud platform. A total of 300 MapReduce tasks of different types and workloads were run, including WordCount, TeraSort, and PageRank. The collected operating status data and resource usage data are collected with a sampling period of 5 seconds, and finally 30,000 sets of data are obtained for building a multi-modal neural network.

[0100] Finally, in order to verify the effect of the system, a WordCount task with a calculation amount of 400GB and a completion time limit of 1700 seconds was submitted to a cluster with an initial cluster size of 16 virtual machines. like Figure 6 As shown, AS-M and AS-R in the legend represent the running status of the Map process and the Reduce process of th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and system for expanding elastic resources based on machine learning, which belong to the field of cloud computing technology and the field of deep learning, including: d In the case of the calculation amount of the task, the regression model is used to calculate the minimum total amount of resources required to complete the task; the current running status and resource utilization of the task are continuously collected during the running of the task, and the minimum total amount of resources, the current running status of the task, and The resource utilization rate and task calculation amount are input into the prediction model for prediction, and the task completion time T is obtained c ; if T c >t d , then the calculation makes the completion time T of the final task c '<t d The minimum total amount of resources; if the task is not completed, the collection will continue, and if the task is completed, the collection will be stopped. The method of the invention calculates the minimum amount of resources through a regression model to ensure that tasks can be completed on time, predicts the completion time during operation, and automatically performs elastic expansion of computing resources when the completion time exceeds the operation deadline.

Description

technical field [0001] The invention belongs to the field of cloud computing technology and the field of deep learning, and more specifically relates to a method and system for expanding elastic resources based on machine learning. Background technique [0002] The cloud computing business model is generally that the tenant first tells the cloud service provider the number of cloud computing resources that need to be applied, and then the cloud service provider will allocate these resources according to the tenant's request. In this mode, tenants need to estimate the total amount of resources needed according to their own business. However, because cloud tenants lack the understanding of the underlying implementation of cloud service provider services, it is difficult for them to estimate the amount of computing resources required in the virtual environment of the cloud platform based on their previous experience of running business locally. Therefore, the tenant proposes a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/455G06F9/50
CPCG06F9/45558G06F9/5072G06F2009/45595
Inventor 刘方明金海李羿
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products