Resource management method of data center based on statistic model in cloud computing environment

A cloud computing environment and statistical model technology, applied in the direction of resource allocation, multi-programming devices, etc., can solve problems that cannot represent the overall data, cannot handle workload, and resource management is inflexible, so as to ensure accuracy and improve resource supply Efficiency, cost-saving effects

Inactive Publication Date: 2011-04-06

SHANGHAI JUNESH INFORMATION TECH CO LTD

4 Cites 142 Cited by

AI-Extracted Technical Summary

Problems solved by technology

The data collected in this way cannot represent the overall data in the production operating environment, and cannot handle the changing workload in the cloud computing environment

This method is inflexible in t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Method used

See Fig. 2 based on statistical resource management model, the present invention adopts real-time load and resource monitoring mode, analyzes and processes monitoring data by statistical model, generates ideal distribution data, ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Abstract

The invention relates to a resource management method cluster of a data center based on the statistics in cloud computing environment, which comprises four steps of: 1. collecting work loads, application problem performance and resource use condition information; 2. inputting the information into a model and carrying out model analysis by using a statistic analysis method KCCA (Kernel Canonical Correlation Analysis) and remote-distance relative algorithm; 3. classifying work modes according to the current environment, and regulating the resource distribution according to control parameters; and 4. regulating the resource distribution according to the controller output and simultaneously updating the resource states. The resource management method of the invention firstly considers the characteristic that the work loads of the novel data center in the cloud computing environment are continuously changed, the real-time monitoring and the elastic management are carried out on the resources of the data center according to user requirements and the resource use conditions to ensure that the integral resource consumption can reach the minimum under the condition that the system performance is not influenced. The resource management method has wide practical value and application prospects in the technical field of elastic resource management of the cloud computing data center.

Application Domain

Resource allocation

Technology Topic

Storage resource managementResource distribution +16

Image

Examples

Experimental program(1)

Example Embodiment

[0030] In order to make the objectives, technical solutions and advantages of the present invention more clearly expressed, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] In terms of hardware conditions, the present invention requires each node to support shared storage at the same time. In terms of software conditions, if the operating system uses Linux, the kernel version is required to be above 2.6.18 to avoid the defects of low-version kernel in power management.

[0032] The equipment conditions that the present invention needs to meet are shown in figure 1 , The data center infrastructure includes: the main control node is interconnected with the data network through a network adapter, monitors and controls multiple nodes and multiple virtual machines in the node, the main controller is responsible for unified deployment and management of downstream node controllers, including adding , Delete and migrate any number of physical drives and storage media for readable data in the control system. The management model is responsible for analyzing and processing the collected load and resource usage information, and then handing it over to the controller for control. A computing node contains any number of virtual machines, and each virtual machine is installed with an application (such as Web2.0). Each node contains a node controller responsible for virtual machine resource control within the node, and a resource driver is responsible for resource allocation Management, multiple virtual machines and virtual machine monitors including application resource usage monitors and performance monitors.

[0033] Statistics-based resource management model see figure 2 , The present invention adopts real-time load and resource monitoring methods, analyzes and processes monitoring data through statistical models, generates ideal distribution data, and processes them in combination with actual conditions through a controller, and outputs actual distribution data to guide system resource distribution to ensure that fewer resources are used Under the circumstances, the maximum performance of the system is achieved while saving system overhead.

[0034] The information collected by the data center includes: the corresponding application load information of the data center, take the number of requests per second as an example, analyze the user demand for the application; collect the user demand indicators involved in the user service level agreement SLA, such as Throughput rate and response time are used as system performance metrics; data center resource usage, such as CPU usage, I/O, and network bandwidth, are used to measure system usage and overall system capacity. It is necessary to collect the above-mentioned information in real time and update the data in time to ensure the quality of data center service.

[0035] An example is described below, such as Figure 5 As shown, including the following steps:

[0036] Step 501: Collect application program load, performance and resource usage information system in real time. At the same time, the acquired information is recorded, and the update period is set to 1.0 second.

[0037] Step 502: Take the information collected in the above steps as input and input it into the statistical model module for analysis, mainly analyzing the relationship between application load, performance and resource usage, and using its output as the input of step 503. Specific as Image 6 The statistical model analysis module is shown.

[0038] Step 503: Controller mode analysis. According to the virtual machine resources required by the user, such as CPU, memory, etc., use the resource allocation plan given by the above statistical model, consider the interference of the external environment on the system (such as system management and maintenance requirements), according to the current analysis of the working mode , Adopt different control modes. In different modes, the controller hysteresis parameters are different. For example, in the major holiday mode, setting α=0.9 can quickly respond to system abnormalities and quickly add resources, β=0.01 ensures that the system can remove resources when needed A more conservative estimate does not require rapid response. This set of parameters is suitable for modes that require fast response. In other modes, appropriately adjust this set of parameters to make the system react to changes in workload to meet user needs without consuming excessive resources.

[0039] Step 504: Control the allocation of resources according to the parameters output by the controller. First, the main controller assigns tasks to the node controller, and the node controller allocates virtual machines that meet the requirements according to specific application requirements.

[0040] The statistical model analysis module process is as follows Image 6 As shown, including the following steps:

[0041] Step 601: During system operation, according to Figure 5 The information collected in step 501 is to monitor the curve generated by the system parameters in real time, and observe whether the curve has abrupt points or abnormal values that do not conform to the fitted curve. If it exists, it means that the current data center has a rush hour for the application, and step 602 should be taken for processing; if it does not exist, it means that the system is operating stably, and step 603 is performed.

[0042] Step 602: An abnormal value point appears on the system curve, which proves that the resources required by the system need to be changed significantly. At this time, it is necessary to quickly analyze the usage of system resources, and whether the resource requirements can be met when the system resources reach the maximum capacity. Because it is an outlier point, the current obtained parameters cannot represent the overall load, performance and resource relationship model, but if it is not processed in time, it will have a great impact on the performance of the system, so it is necessary to analyze the outlier points in time deal with. If the system resource pool resources meet the current demand, they will be directly handed over to the controller for processing to quickly resolve the current outlier points; if the current system resources reach the maximum capacity, the application performance requirements cannot be met, and virtualization technologies, such as virtual Quickly add resources such as machine migration to meet user resource needs. Supply on demand, saving system resource overhead. Put the added resources into the system resource pool and control them uniformly through the controller.

[0043] Step 603: Use a simple linear regression model to predict the next 5-minute workload, that is, the number of application requests per second. A simple linear regression model can effectively capture the changes in workload over time, and even more complex historical data can easily summarize and predict its load.

[0044] Step 604: The predicted workload is used as the input of the model to evaluate the resource requirements required by the existing workload and the performance that the system can achieve. Many complex factors will affect the performance of the application, such as mixed load, application code changes, etc., using the KCCA algorithm and long-distance related algorithms to achieve multivariate statistical analysis and modeling, and analyze the impact of multiple influencing factors on system performance. Influence, adjust model parameters in real time, and generate ideal resource allocation data. Finally, output it as Figure 5 The input of step 503 jumps to the controller mode analysis.

[0045] image 3 It is a schematic diagram of the model-based resource management process of the present invention; Figure 4 It is a schematic diagram of controller control resource allocation.

[0046] In this example, when viewing parameters such as resource usage and workload and updating the corresponding parameters, it is executed cyclically according to the resource collection cycle during the resource allocation process. Using the above-mentioned flexible resource management method can ensure that the system uses less resources to achieve maximum performance at any time, so as to meet the needs of users.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that the present invention can still be modified Or equivalent replacements, any modifications or partial replacements that do not depart from the spirit and scope of the present invention shall be covered by the scope of the claims of the present invention.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Login to view more

PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Login to view more

Classification and recommendation of technical efficacy words

Reduce overhead

People also interested in

Guaranteed accuracy User needs Workload Analysis method Statistical analysis Computer science Cloud computing

Resource management method of data center based on statistic model in cloud computing environment

AI-Extracted Technical Summary

Problems solved by technology

Method used

Abstract

Image

Examples

Example Embodiment

PUM

Description & Claims & Application Information

Similar technology patents

Method and apparatus for improving file access performance of distributed storage system

Method for transition between distributed transmission and localized transmission

User terminal safety communication method in platform of Internet of Things

Monitor Chaining and Docking Mechanism

Preamble design method for reducing overhead during handover in hierarchical cellular system

Classification and recommendation of technical efficacy words

Low overhead fault tolerance through hybrid checkpointing and replay

Non-volatile memory and method with block management system

Method and system for providing secure, centralized access to remote elements

Downhole data transmission system

Row-diagonal parity technique for enabling efficient recovery from double failures in a storage array

People also interested in