Machine learning-based elastic resource expansion method and system

A technology of machine learning and extension method, which is applied in the field of cloud computing, can solve the problems of low estimation accuracy and difficulty in elastic resource management, and achieve high accuracy, solve the effects of computing speed reduction and elastic resource management

Active Publication Date: 2019-09-06
HUAZHONG UNIV OF SCI & TECH
View PDF8 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for expanding elastic resources based on machine learning, thereby solving the technical problems of difficult elastic resource management and low estimation accuracy in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Machine learning-based elastic resource expansion method and system
  • Machine learning-based elastic resource expansion method and system
  • Machine learning-based elastic resource expansion method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0099] In order to verify the feasibility and effectiveness of the method of the present invention, the method of the present invention is verified in a real environment. Experimental preparations include: setting up a cluster with a maximum number of 40 virtual machines on the Alibaba Cloud platform. A total of 300 MapReduce tasks of different types and workloads were run, including WordCount, TeraSort, and PageRank. The collected operating status data and resource usage data are collected with a sampling period of 5 seconds, and finally 30,000 sets of data are obtained for building a multi-modal neural network.

[0100] Finally, in order to verify the effect of the system, a WordCount task with a calculation amount of 400GB and a completion time limit of 1700 seconds was submitted to a cluster with an initial cluster size of 16 virtual machines. Such as Figure 6 As shown, AS-M and AS-R in the legend represent the running status of the Map process and the Reduce process of...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a machine learning-based elastic resource expansion method and system, and belongs to the technical field of cloud computing and the field of deep learning, and the method comprises the steps: calculating the minimum total resource amount required for completing a task by utilizing a regression model under the condition that the operation cut-off time limit td and the taskcalculation amount of the to-be-operated task are known; continuously collecting the current running state and the resource utilization rate of the task in the running process of the task, and inputting the minimum total resource amount, the current running state of the task, the resource utilization rate and the task calculation amount into a prediction model for prediction to obtain the task completion time Tc; if Tc is larger than td, calculating the minimum total resource amount enabling the final task completion time Tc 'to be smaller than td; and if the task is not completed, continuingcollection, and if the task is completed, stopping collection. According to the method, the minimum total resource amount is calculated through the regression model to ensure that the task can be completed on time, the completion time is predicted in the operation process, and when the completion time exceeds the operation cut-off time limit, elastic expansion of the calculation resources is automatically carried out.

Description

technical field [0001] The invention belongs to the field of cloud computing technology and the field of deep learning, and more specifically relates to a method and system for expanding elastic resources based on machine learning. Background technique [0002] The cloud computing business model is generally that the tenant first tells the cloud service provider the number of cloud computing resources that need to be applied, and then the cloud service provider will allocate these resources according to the tenant's request. In this mode, tenants need to estimate the total amount of resources needed according to their own business. However, because cloud tenants lack the understanding of the underlying implementation of cloud service provider services, it is difficult for them to estimate the amount of computing resources required in the virtual environment of the cloud platform based on their previous experience of running business locally. Therefore, the tenant proposes a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/455G06F9/50
CPCG06F9/45558G06F9/5072G06F2009/45595
Inventor 刘方明金海李羿
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products