Method, medium and system for establishing virtualized server load response model

Abstract

The present invention provides a method, medium and system for establishing a virtualized server load response model, which belongs to the field of virtualized server load response technology. By setting initial test conditions and changing them multiple times, multiple groups of load tests are performed on the virtualized server, and test data under different conditions are obtained to form a load data set. A load response characteristic equation group containing all parameters is established, and the data set is used for fitting. Then, a resource conversion model and a resource competition model are established to reflect the resource mapping relationship between the physical server and the virtual server and the resource competition between the virtual servers. The two models are integrated, and the fitted equation group is solved to obtain a numerical solution matrix. Finally, a virtualized server load response model is constructed based on the matrix to describe the resource usage and performance of the virtual server under different load conditions, which solves the technical problem that the server load response model established by the prior art is not accurate enough.

Description

Technical Field

[0001] The present invention belongs to the technical field of virtualized server load response, and in particular, relates to a method, medium and system for establishing a virtualized server load response model. Background Art

[0002] Virtualization technology, as the foundation for implementing new IT architectures such as cloud computing and containerization, plays an increasingly important role in the current information technology landscape. By abstracting physical computing resources into virtual resources, virtualization significantly improves the resource utilization and flexibility of IT systems and is widely used in numerous application scenarios, including web services, databases, and big data processing. However, in a virtualized environment, physical server resources such as CPU, memory, storage, and network must be shared by multiple virtual machines. This leads to intense resource competition between virtual machines, impacting their performance. Furthermore, due to the overhead of the virtualization layer itself and the inefficiency of converting physical resources to virtual resources, the actual resource usage and performance metrics of virtual machines differ significantly from those of physical servers. Therefore, accurately modeling and predicting the resource usage and performance of virtualized servers under different workloads has become a key technical challenge that urgently needs to be addressed.

[0003] To address these issues, existing technologies primarily utilize metrics-based monitoring methods. These methods deploy monitoring agents to collect real-time CPU, memory, storage, and network usage data for virtual and physical machines, and then analyze and predict these data using visualization tools. While this approach can provide a direct view of resource usage in a virtualized environment, it struggles to deeply analyze the impact of various factors on performance, and the resulting server load response models lack sufficient accuracy. Summary of the Invention

[0004] In view of this, the present invention provides a method, medium and system for establishing a virtualized server load response model, which can solve the technical problem that the server load response model established in the prior art is not accurate enough.

[0005] The present invention is achieved in that:

[0006] A first aspect of the present invention provides a method for establishing a virtualized server load response model, comprising the following steps:

[0007] S10. Setting initial test conditions, performing load testing on the virtualized server, and obtaining test data;

[0008] S20, changing the test conditions multiple times, performing multiple load tests, obtaining test data under different conditions, and merging each set of test conditions and corresponding test data into a virtualized server load data set;

[0009] S30, establishing a load response characteristic equation group including all parameters in the virtualized server load data set;

[0010] S40, fitting the virtualized server load data set to the load response characteristic equation group to obtain a fitted equation group;

[0011] S50, establishing a resource conversion model based on the resource mapping relationship between the physical server and the virtual server;

[0012] S60, establishing a resource competition model based on resource competition and interference relationships among multiple virtual servers within the same physical server;

[0013] S70, integrating the resource conversion model and the resource competition model, solving the fitted equation group, and obtaining a numerical solution matrix of the virtualized server load response characteristics;

[0014] S80. Based on the numerical solution matrix, a virtualized server load response model is constructed to describe the resource usage and performance of the virtual server under different load conditions.

[0015] On the basis of the above technical solution, the method for establishing a virtualized server load response model of the present invention can also be improved as follows:

[0016] The test conditions include physical server configuration, virtualization platform type, number of virtual machines, service type, network topology and initial resource allocation.

[0017] Furthermore, the test data includes CPU usage, memory usage, storage I / O, network throughput, response time and energy consumption.

[0018] Furthermore, the load response characteristic equation group includes a computing resource equation, a storage resource equation, a network resource equation, a service performance equation, an energy consumption equation, a virtualization overhead equation and a resource scheduling equation.

[0019] Computational resource equation:

[0020] Where C u is the CPU usage; V cpu is the number of virtual CPU cores; P cpu is the number of physical CPU cores; L is the load intensity; N is the number of virtual machines; α1, α2, α3 are unknown coefficients; ε1 is the error term.

[0021] Storage resource equation:

[0022] Where, I o is the storage I / O rate; Vstorage is the virtual storage capacity; P storage is the physical storage capacity; R is the read operation frequency; W is the write operation frequency; β1, β2, β3 are unknown coefficients; ε2 is the error term.

[0023] Network resource equation:

[0024] Where, T p is the network throughput; V bandwidth is the virtual network bandwidth; P bandwidth is the physical network bandwidth; P s is the number of packets sent; P r is the number of received data packets; γ1, γ2, γ3 are unknown coefficients; ε3 is the error term.

[0025] Business Performance Equation: R t =δ1·C u +δ2·I o +δ3·T p +δ4·Q+ε4;

[0026] Where R t is the response time; Q is the number of concurrent requests; δ1, δ2, δ3, δ4 are unknown coefficients; ε4 is the error term.

[0027] Energy consumption equation: E c =η1·C u +η2·I o +η3·T p +η4·T+ε5;

[0028] Where, E c is the energy consumption; T is the running time; η1, η2, η3, η4 are unknown coefficients; ε5 is the error term.

[0029] Virtualization overhead equation:

[0030] Where, O v is the virtualization overhead; V total is the total virtual resource; P total is the total physical resources; S f is the resource switching frequency; θ1, θ2, θ3 are unknown coefficients; ε6 is the error term.

[0031] Resource scheduling equation:

[0032] Where S e is the scheduling efficiency; A r Allocated resources; R r To request resources; B ris the resource balance; M t is the number of migrations; λ1, λ2, λ3 are unknown coefficients; ε7 is the error term.

[0033] Furthermore, the method for fitting the virtualized server load data set to the load response characteristic equation group is the least square method.

[0034] Furthermore, the resource conversion model is used to describe the conversion process from physical resources to virtual resources and its efficiency.

[0035] Furthermore, the resource competition model is used to describe the mutual influence between virtual servers.

[0036] Resource conversion model: V r =φ(P r ,H t ,V o )=κ1·P r ·(1-H t )-κ2·V o +ε8;

[0037] Where V r is a virtual resource; P r For physical resources; H t is the virtualization layer overhead ratio; V o is the resource virtualization loss; κ1, κ2 are conversion coefficients; ε8 is the error term.

[0038] Resource competition model: C f =ψ(R d ,I l ,P c )=μ1·R d +μ2·I l +μ3·P c +ε9;

[0039] Where C f is the competitive factor; R d is the resource demand; I l is the interference level; P c is the priority coefficient; μ1, μ2, μ3 are weight coefficients; ε9 is the error term.

[0040] Numerical solution matrix: The numerical solution matrix M is specifically expressed as follows:

[0041]

[0042] Where n is the number of test data sets, and each row represents the load response characteristics of a set of test data. This matrix is ​​used to describe the performance and resource usage of virtualized servers under different load conditions.

[0043] Virtualized server load response model:

[0044] P m =f(L,R,E,C)=ω1·L+ω2·R+ω3·E+ω4·C+ε 10 ;

[0045] Where, P m is the performance index; L is the load vector; R is the resource vector; E is the environment vector; C is the configuration vector; ω1, ω2, ω3, ω4 are weight coefficients; ε 10 is the error term.

[0046] Parameter acquisition method:

[0047] Load vector L: Use a load generator to simulate workloads of different types and intensities and record CPU, memory, storage, and network usage.

[0048] Resource vector R: collects resource usage data of virtual machines and physical machines in real time through monitoring tools.

[0049] Environment vector E: includes information such as virtualization platform type and network topology, and is obtained through configuration files or system APIs.

[0050] Configuration vector C: includes the number of virtual machines, resource allocation policy, etc., and is obtained through the virtualization management platform interface.

[0051] Steps for determining weight coefficients ω1, ω2, ω3, ω4:

[0052] Step 1: Collect a large amount of experimental data, including performance indicators under different load, resource, environment, and configuration conditions.

[0053] Step 2: Use the multiple linear regression analysis method to calculate the influence of each vector on the performance index based on the collected data.

[0054] Step 3: Solve the regression equation using the least squares method to obtain the optimal estimate of the weight coefficient.

[0055] Step 4: Use cross-validation to evaluate the accuracy and generalization ability of the model and adjust the weight coefficients if necessary.

[0056] Furthermore, the resource conversion model and the resource competition model are integrated to solve the fitted equation group, specifically: the resource conversion model and the resource competition model are embedded in the fitted load response characteristic equation group for solving.

[0057] Specifically, step S10 includes the following sub-steps:

[0058] Step 101: Determine the specification parameters of the physical server's hardware resources, such as CPU, memory, storage, and network, as initial condition inputs.

[0059] Step 102: Select the type of virtualization platform to be used, such as VMware, Hyper-V, or KVM, and record this information.

[0060] Step 103: Determine the number of virtual machines to be initially deployed, usually starting with one and gradually increasing to a reasonable range to cover different load intensities.

[0061] Step 104: Determine the business type, such as Web service, database service, or big data application, which determines the characteristics of the workload.

[0062] Step 105: Describe the network topology of the virtualized environment, including the connection between the virtual machines and the physical machines.

[0063] Step 106: Set an initial resource allocation strategy, such as fixed allocation, dynamic allocation, or priority allocation.

[0064] Step 107: Use a specialized load testing tool to apply workloads of various types and intensities to the virtualized server, and continuously record indicators such as CPU usage, memory usage, storage I / O, network throughput, response time, and energy consumption.

[0065] Specifically, step S20 includes the following sub-steps:

[0066] Step 201: Keep the physical server configuration unchanged and gradually adjust parameters such as virtualization platform type, number of virtual machines, service type, and initial resource allocation strategy.

[0067] Step 202: According to the test requirements, the range of load intensity is appropriately expanded or narrowed to cover different workload conditions.

[0068] Step 203: For each set of test conditions, repeatedly perform the load test and collect indicator data such as CPU usage, memory usage, storage I / O, network throughput, response time, and energy consumption.

[0069] Step 204: Combine each set of test conditions and corresponding test data into a virtualized server load data set, and record the specific parameter configuration of each data set.

[0070] Specifically, step S30 includes the following sub-steps:

[0071] Step 301: Analyze the relationship between the key performance indicators of the virtualized server and various resource indicators based on the collected test data.

[0072] Step 302: For indicators such as CPU usage, memory usage, storage I / O, network throughput, response time and energy consumption, respectively establish computing resource equations, storage resource equations, network resource equations, service performance equations, energy consumption equations and virtualization overhead equations.

[0073] Step 303: In these equations, influencing factors such as the number of virtual CPU cores, the number of physical CPU cores, load intensity, the number of virtual machines, read and write operation frequency, the number of concurrent requests, and the running time are introduced.

[0074] Step 304: Use a multiple linear regression analysis method to estimate the undetermined coefficients in each equation.

[0075] Step 305: In the resource scheduling equation, factors such as allocated resources, requested resources, resource balance, and migration times are considered to describe the resource scheduling efficiency of the virtualized server.

[0076] Specifically, step S40 includes the following sub-steps:

[0077] Step 401: Use the least square method to perform parameter fitting on the aforementioned equation group, using the various indicator data obtained from the test data set as independent variables and dependent variables.

[0078] Step 402: Solve the undetermined coefficients in each equation through an iterative optimization algorithm so that the equation group can best fit the entire load data set.

[0079] Step 403: Perform statistical tests on the fitting results, including testing indicators such as goodness of fit, significance level, and residual distribution, to ensure the predictive ability of the equation system.

[0080] Step 404: If necessary, the equation form may be further adjusted or other influencing factors may be introduced to improve the fitting effect.

[0081] Specifically, step S50 includes the following sub-steps:

[0082] Step 501: Analyze how physical resources such as CPU, memory, storage, and network bandwidth are converted into virtual resources for use by virtual machines.

[0083] Step 502: Consider the resource consumption factor of the virtualization layer and the resource loss caused by the limitation of the virtualization technology itself.

[0084] Step 503: Establish resource conversion model V r =φ(P r ,H t ,V o ), where V r is a virtual resource, P r For physical resources, Ht is the virtualization layer overhead ratio, V o This is the loss of resource virtualization.

[0085] Step 504: Determine conversion coefficients κ1 and κ2 to describe the conversion process from physical resources to virtual resources and its efficiency.

[0086] Specifically, step S60 includes the following sub-steps:

[0087] Step 601: Analyze the interference effect caused by resource demand competition between virtual machines.

[0088] Step 602: Introduce factors such as resource demand, interference level, and priority coefficient to describe resource competition between virtual machines.

[0089] Step 603: Establish resource competition model C f =ψ(R d ,I l ,P c ), where C f is the competitive factor, R d is the resource demand, I l is the interference level, P c is the priority coefficient.

[0090] Step 604: Determine weight coefficients μ1, μ2, and μ3 to quantitatively describe the mutual influence between virtual machines.

[0091] Specifically, step S70 includes the following sub-steps:

[0092] Step 701: embed the resource conversion model and the resource competition model into each load response characteristic equation.

[0093] Step 702: Use a numerical solution method such as the Newton-Raphson method or an iterative method to jointly solve the system of equations.

[0094] Step 703: During the solution process, the coupling relationship and constraints between the equations need to be considered.

[0095] Step 704: Finally, a numerical solution matrix M including indicators such as CPU usage, storage I / O, network throughput, response time, energy consumption, virtualization overhead, and resource scheduling efficiency is obtained.

[0096] Specifically, step S80 includes the following sub-steps:

[0097] Step 801: Analyze the relationship between various indicator data recorded in the numerical solution matrix M and load characteristics.

[0098] Step 802: Establish a virtualized server load response model P m =f(l,R,E,C), where P m is the performance indicator, L is the load vector, R is the resource vector, E is the environment vector, and C is the configuration vector.

[0099] Step 803: Use a multiple linear regression analysis method to determine the degree of influence of each vector on the performance index, that is, the weight coefficients ω1, ω2, v3 and v4.

[0100] Step 804: Solve the regression equation using the least squares method to obtain the optimal estimate of the weight coefficient.

[0101] Step 805: Use the cross-validation method to evaluate the accuracy and generalization ability of the model, and adjust the weight coefficient if necessary.

[0102] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run in a computer, they are used to execute the above-mentioned method for establishing a virtualized server load response model.

[0103] A third aspect of the present invention provides a virtualized server load response model establishment system, which includes the above-mentioned computer-readable storage medium.

[0104] Compared with the prior art, the method, medium, and system for establishing a virtualized server load response model provided by the present invention have the following beneficial effects:

[0105] 1. Comprehensiveness: This paper establishes a set of equations that include key performance indicators such as CPU utilization, memory usage, storage I / O, network throughput, response time, energy consumption, virtualization overhead, and resource scheduling efficiency. It can comprehensively describe the resource consumption and performance of virtualized servers under different load conditions.

[0106] 2. Accuracy: The invented method determines the quantitative relationship between various performance indicators and influencing factors through fitting and optimization of a large amount of measured data, and can accurately predict the load response characteristics of the virtualized server.

[0107] 3. Explainability: The invented method is based on the resource conversion model and the resource competition model, and deeply analyzes the mapping relationship from physical resources to virtual resources in the virtualization environment, as well as the inherent mechanism of resource competition between virtual machines, and has strong explainability.

[0108] 4. Adaptability: The invented method only needs to collect a small amount of configuration information and performance data of the virtualization environment to build a targeted load response model. It has strong adaptability and can be easily applied to actual virtualization system management and optimization.

[0109] 5. Practicality: The constructed load response model can provide strong support for dynamic resource scheduling, performance prediction and bottleneck diagnosis of virtualization systems, and has broad application prospects in new IT architectures such as cloud computing and containerization.

[0110] In general, the method for establishing a virtualized server load response model proposed in the present invention can effectively make up for the shortcomings of the existing technology and solve the technical problem that the server load response model established by the existing technology is not accurate enough. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 A flow chart of the method provided by the present invention;

[0112] Figure 2 A comparison chart of resource usage in the Examples section;

[0113] Figure 3 This is a performance load relationship diagram of the embodiment part;

[0114] Figure 4 Resource allocation heat map for the example section;

[0115] Figure 5 It is a time series diagram of performance indicators in the embodiment section;

[0116] Figure 6 This is a resource competition network diagram of the embodiment section. DETAILED DESCRIPTION

[0117] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0118] like Figure 1 FIG. 1 is a flow chart of a method for establishing a virtualized server load response model provided by the present invention. The method includes the following steps:

[0119] S10. Set initial test conditions, perform load testing on the virtualized server, and obtain test data. The test conditions include physical server configuration, virtualization platform type, number of virtual machines, service type, network topology, and initial resource allocation; the test data includes CPU usage, memory usage, storage I / O, network throughput, response time, and energy consumption.

[0120] S20, changing the test conditions multiple times, performing multiple load tests, obtaining test data under different conditions, and merging each set of test conditions and corresponding test data into a virtualized server load data set;

[0121] S30, establishing a load response characteristic equation group containing all parameters in the virtualized server load data set, including a computing resource equation, a storage resource equation, a network resource equation, a service performance equation, an energy consumption equation, a virtualization overhead equation, and a resource scheduling equation;

[0122] S40, fitting the virtualized server load data set to the load response characteristic equation group to obtain a fitted equation group;

[0123] S50, establishing a resource conversion model based on the resource mapping relationship between the physical server and the virtual server, for describing the conversion process from physical resources to virtual resources and its efficiency;

[0124] S60, based on the resource competition and interference relationship between multiple virtual servers within the same physical server, establish a resource competition model to describe the mutual influence between the virtual servers;

[0125] S70, integrating the resource conversion model and the resource competition model, solving the fitted equation group, and obtaining a numerical solution matrix of the virtualized server load response characteristics;

[0126] S80. Based on the numerical solution matrix, a virtualized server load response model is constructed to describe the resource usage and performance of the virtual server under different load conditions.

[0127] The specific implementation of the above steps is described in detail below:

[0128] The step S10 includes the following sub-steps:

[0129] Step 101: Determine the specifications of the physical server's CPU, memory, storage, and network hardware resources as initial condition inputs. These parameters are denoted as P cpu 、P memory 、P storage and P bandwidth The hardware configuration of the physical server directly affects the availability of virtualized resources and is the basis for establishing a resource conversion model.

[0130] Step 102: Select the type of virtualization platform to be used, such as VMware, Hyper-V, or KVM, and record this information. Different virtualization platforms have different resource management and scheduling strategies, which will affect the performance of virtual machines and therefore need to be taken into account in the model. type Indicates the virtualization platform type.

[0131] Step 103: Determine the number of VMs N to initially deploy. Typically, start with one VM and gradually increase the number to a reasonable range to cover varying load intensities. The number of VMs is a key factor affecting resource contention and needs to be reflected in the model.

[0132] Step 104: Determine business type B type , such as web services, database services, or big data applications, which determine the characteristics of the workload. Different business types will have different requirements for server resources and need to be distinguished in the model.

[0133] Step 105: Describe the network topology N of the virtualized environment topo , including the connection between virtual machines and physical machines.,Network topology affects resource competition and interference between virtual machines,,and is an important factor in resource competition modeling.

[0134] Step 106: Set the initial resource allocation strategy R init , such as fixed allocation, dynamic allocation or priority allocation, etc. The resource allocation policy directly determines the amount of resources that the virtual machine can obtain and needs to be used as an input parameter of the model.

[0135] Step 107: Use a dedicated load testing tool to apply workloads L of various types and intensities to the virtualized server and continuously record the CPU usage C. u 、Memory usage M u , Storage I / O Rate I o , network throughput T p , response time R t and energy consumption E c These test data will constitute the virtualized server load data set.

[0136] Among them, the load intensity L can be expressed as:

[0137] L={L1,L2,…,L n};

[0138] Among them, L i Represents the load intensity of the i-th type.

[0139] Step 108: Set the initial condition parameter P cpu 、P memory 、P storage 、P bandwidth 、V type , N, B type 、N topo 、R init and test data C u 、M u , I o 、Tp 、R t 、E c The datasets are combined into an initial dataset D0, which describes the performance characteristics of the virtualized servers under initial conditions.

[0140] Through the above steps, load test data of the virtualized server under initial conditions is obtained. This data reflects the load response characteristics of the virtualized server under specific configuration and environment.

[0141] Next, in step S20, it is necessary to change the initial conditions multiple times, perform multiple load tests, and obtain test data under different conditions. Specifically, it includes the following sub-steps:

[0142] Step 201: For i=1, 2, ..., m, where m is the number of test groups:

[0143] (1) Keep the physical server configuration unchanged and gradually adjust the virtualization platform type Number of virtual machines N (i) , business type and initial resource allocation strategy and other parameters.

[0144] (2) According to the test requirements, adjust the load intensity range appropriately

[0145] (3) For each set of test conditions Repeat the load test k times and collect CPU usage Memory usage Storage I / O rate Network throughput Response time and energy consumption And other indicator data, where j = 1, 2,…, k.

[0146] Step 202: Combine each set of test conditions and corresponding test data into a virtualized server load data set.

[0147]

[0148] By adjusting the initial conditions and performing load tests multiple times, we obtain a virtualized server load dataset D={D0,D1,D2,…,D m These data will provide the basis for subsequent model building.

[0149] In step S30, it is necessary to establish a load response characteristic equation group that includes all parameters in the virtualized server load data set. Specifically, it includes the following sub-steps:

[0150] Step 301: Analyze the relationship between the key performance indicators of the virtualized server and various resource indicators based on the collected test data.

[0151] Step 302: CPU usage C u 、Memory usage M u , Storage I / O Rate I o , network throughput T p , response time R t and energy consumption E c Based on the indicators, the following computing resource equations, storage resource equations, network resource equations, service performance equations, energy consumption equations, and virtualization overhead equations are established:

[0152] Computational resource equation:

[0153]

[0154] Among them, V cpu is the number of virtual CPU cores, P cpu is the number of physical CPU cores, L is the load intensity, N is the number of virtual machines, α1, α2, and α3 are unknown coefficients, and ε1 is the error term. This equation describes the relationship between CPU utilization and the number of virtual CPU cores, load intensity, and the number of virtual machines.

[0155] Storage resource equation:

[0156]

[0157] Among them, V storage is the virtual storage capacity, P storage is the physical storage capacity, R is the read operation frequency, W is the write operation frequency, β1, β2, and β3 are unknown coefficients, and ε2 is the error term. This equation describes the relationship between the storage I / O rate, the virtual storage capacity, and the read and write operation frequencies.

[0158] Network resource equation:

[0159]

[0160] Among them, V bandwidth is the virtual network bandwidth, P bandwidth is the physical network bandwidth, P s is the number of packets sent, P r is the number of received packets, γ1, γ2, and γ3 are unknown coefficients, and ε3 is the error term. This equation describes the relationship between network throughput and virtual network bandwidth, as well as the number of packets sent and received.

[0161] Business Performance Equation:

[0162] R t=δ1·C u +δ2·I o +δ3·T p +δ4·Q+ε4;

[0163] Where Q is the number of concurrent requests, δ1, δ2, δ3, and δ4 are undetermined coefficients, and ε4 is the error term. This equation describes the relationship between response time and CPU utilization, storage I / O rate, network throughput, and the number of concurrent requests.

[0164] Energy consumption equation:

[0165] E c =η1·C u +η2·I o +η3·T p +η4·T+ε5;

[0166] Where T is the runtime, η1, η2, η3, and η4 are unknown coefficients, and ε5 is the error term. This equation describes the relationship between energy consumption and CPU utilization, storage I / O rate, network throughput, and runtime.

[0167] Virtualization overhead equation:

[0168]

[0169] Among them, V total is the total virtual resource, P total is the total physical resource, S f is the resource switching frequency, θ1, θ2, θ3 are unknown coefficients, and ε6 is the error term. This equation describes the relationship between virtualization overhead and the number of virtual machines, virtual resource utilization, and resource switching frequency.

[0170] Resource scheduling equation:

[0171]

[0172] Among them, A r is the allocated resource, R r To request resources, B r is the resource balance, M t is the number of migrations, λ1,λ2,λ3 are unknown coefficients, and ε7 is the error term. This equation describes the relationship between resource scheduling efficiency and allocated resources, resource balance, and the number of migrations.

[0173] By establishing these equations, we can fully describe the resource consumption and performance of virtualized servers under different load conditions. These equations will provide the basis for subsequent model solutions.

[0174] Step 303: Introduce influencing factors such as the number of virtual CPU cores V into the above equation.cpu , Number of physical CPU cores P cpu , load intensity L, number of virtual machines N, read and write operation frequencies R and W, number of concurrent requests Q, running time T, etc. These factors describe the hardware configuration, workload characteristics and business requirements of the virtualization server.

[0175] Step 304: Use multiple linear regression analysis to estimate the unknown coefficients α1, α2, α3, β1, β2, β3, γ1, γ2, γ3, δ1, δ2, δ3, δ4, η1, η2, η3, η4, θ1, θ2, θ3, λ1, λ2, and λ3 in each equation. These coefficients reflect the degree of influence of each factor on the corresponding indicator.

[0176] Step 305: The above equations form a complete virtualized server load response characteristic model. This model covers key indicators such as CPU utilization, memory usage, storage I / O rate, network throughput, response time, energy consumption, virtualization overhead, and resource scheduling efficiency, and can comprehensively describe the performance of the virtualized server under different load conditions.

[0177] In step S40, the virtualized server load data set D needs to be fitted to the aforementioned load response characteristic equations to obtain a fitted equation set. This specifically includes the following sub-steps:

[0178] Step 401: Use the least square method to perform parameter fitting on the aforementioned equations. This requires using various indicator data obtained from the data set D as independent variables and dependent variables.

[0179] For example, for the computational resource equation The following fitting function can be constructed:

[0180]

[0181] Where i=1,2,…,m represents the test group, j=1,2,…,k represents the number of repeated tests. The optimal estimated values ​​of α1, α2 and α3 can be solved.

[0182] Step 402: Solve the undetermined coefficients in each equation by using an iterative optimization algorithm, such as the Newton-Raphson method or the gradient descent method, so that the equation group can best fit the entire load data set D.

[0183] Step 403: Perform statistical tests on the fitting results, including testing indicators such as goodness of fit, significance level, and residual distribution, to ensure the predictive ability of the equation system. If the fitting effect is not ideal, consider adjusting the form of the equation or introducing other influencing factors.

[0184] Step 404: After parameter fitting, a set of equations that can accurately describe the load response characteristics of the virtualized server is obtained. These equations will provide a basis for subsequent resource conversion and resource competition modeling.

[0185] In step S50, a resource conversion model needs to be established based on the resource mapping relationship between the physical server and the virtual server. Specifically, the model includes the following sub-steps:

[0186] Step 501: Analyze how physical resources such as CPU, memory, storage, and network bandwidth are converted into virtual resources for use by virtual machines. The following resource conversion model can be established:

[0187] V r =φ(P r ,H t ,V o )=κ1·P r ·(1-H t )-κ2·V o +ε8;

[0188] Among them, V r For virtual resources, P r For physical resources, H t is the virtualization layer overhead ratio, V o is the resource virtualization loss, κ1 and κ2 are conversion coefficients, and ε8 is the error term.

[0189] Step 502: Consider the resource consumption factor H of the virtualization layer such as the hypervisor or container runtime. t ,This part of the overhead will reduce the actual amount of resources available for the virtual machine. ,At the same time, the resource virtualization loss factor V is introduced o ,Describe the resource loss caused by the limitations of virtualization technology itself.

[0190] Step 503: Establish resource conversion model V r =φ(P r ,H t ,V o ), can describe the conversion process from physical resources to virtual resources and its efficiency. This model will provide a basis for subsequent resource competition modeling.

[0191] Step 504: Determine conversion coefficients κ1 and κ2 to quantitatively describe resource loss during the virtualization process. This requires parameter estimation based on specific virtualization platform characteristics and test data.

[0192] In step S60, a resource competition model needs to be established based on the resource competition and interference relationship between multiple virtual servers within the same physical server. Specifically, it includes the following sub-steps:

[0193] Step 601: Analyze the interference effects caused by resource demand competition between virtual machines, which may cause the performance of the virtual machines to degrade.

[0194] Step 602: Introduce resource demand R d , Interference Level I l and priority coefficient P c Factors such as resource competition between virtual machines can be used to describe the resource competition between virtual machines. The following resource competition model can be established:

[0195] C f =ψ(R d ,I l ,P c )=μ1·R d +μ2·I l +μ3·P c +ε9;

[0196] Among them, C f is the competition factor, μ1, μ2, μ3 are weight coefficients, and ε9 is the error term.

[0197] Step 603: By establishing resource competition model C f =ψ(R d ,I l ,P c ), which can quantitatively describe the mutual influence between virtual machines. This model will provide support for the subsequent load response characteristics solution.

[0198] Step 604: Determine weight coefficients μ1, μ2, and μ3 to quantify the contribution of resource demand, interference level, and priority to resource contention. This requires parameter estimation based on test data and relevant domain knowledge.

[0199] In step S70, it is necessary to integrate the resource conversion model and the resource competition model, solve the above-mentioned fitted equations, and obtain the numerical solution matrix of the virtualized server load response characteristics. Specifically, it includes the following sub-steps:

[0200] Step 701: Convert the resource into model V r =φ(P r ,H t ,V o ) and resource competition model C f =ψ(R d ,I l ,P c ) is embedded into each load response characteristic equation.

[0201] For example, for the computational resource equation You can use V cpu Replace with φ(Pcpu ,H t ,V o ) to reflect the impact of virtualization on CPU resources. At the same time, C f As an adjustment coefficient, to describe the resource competition between virtual machines. The modified equation is:

[0202]

[0203] Step 702: Use a numerical solution method such as the Newton-Raphson method or an iterative method to jointly solve the modified set of equations. This requires considering the coupling relationship and constraints between the equations to ensure the convergence and stability of the solution.

[0204] Step 703: During the solution process, the impact of the resource conversion model and resource competition model on various load response indicators needs to be considered. For example, CPU utilization is affected by both the number of virtual CPU cores and resource competition, while storage I / O rate is affected by virtual storage capacity and resource competition.

[0205] Step 704: Finally, a CPU usage C is obtained. u , Storage I / O Rate I o , network throughput T p , response time R t Energy consumption E c , virtualization overhead O v and resource scheduling efficiency S e The numerical solution matrix M of indicators such as . This matrix completely describes the performance and resource usage of the virtualized server under different load conditions.

[0206] Finally, in step S80, it is necessary to construct a virtualized server load response model based on the above numerical solution matrix M. This specifically includes the following sub-steps:

[0207] Step 801: Analyze the relationship between the various indicator data recorded in the numerical solution matrix M and the load characteristics. The following virtualized server load response model can be established:

[0208] P m =f(L,R,E,C)=ω1·L+ω2·R+ω3·E+ω4·C+ε 10 ;

[0209] Among them, P m is the performance index, L is the load vector, R is the resource vector, E is the environment vector, C is the configuration vector, ω1, ω2, ω3, ω4 are weight coefficients, ε 10 is the error term.

[0210] Step 802: The load vector L includes CPU, memory, storage, and network usage, and can be obtained by simulating workloads of varying types and intensities using a load generator. The resource vector R reflects resource usage data for virtual machines and physical machines, and can be collected in real time using monitoring tools. The environment vector E includes information such as the virtualization platform type and network topology, and can be obtained from configuration files or system APIs. The configuration vector C includes information such as the number of virtual machines and resource allocation policies, and can be obtained through the virtualization management platform interface.

[0211] Step 803: Use multiple linear regression analysis to determine the degree of influence of each vector on the performance index, i.e., the weight coefficients ω1, ω2, ω3, and ω4. This requires collecting a large amount of experimental data, including performance indicators under different load, resource, environment, and configuration conditions.

[0212] Step 804: Solve the regression equation using the least squares method to obtain the optimal estimated values ​​of the weight coefficients. These weight coefficients reflect the relative importance of load, resources, environment, and configuration to the performance of the virtualized server.

[0213] Step 805: Use cross-validation to evaluate the accuracy and generalization ability of the model and adjust the weight coefficients if necessary. This helps ensure that the constructed virtualized server load response model has high prediction accuracy and practical value.

[0214] In summary, the method for establishing a virtualized server load response model proposed in this paper fully utilizes mathematical modeling and numerical analysis techniques, including key modules such as linear regression, resource conversion, and resource competition. By acquiring data from multiple load tests, a set of load response characteristic equations covering various performance indicators is established. Furthermore, the resource conversion and competition mechanisms are integrated to ultimately construct a comprehensive model that can predict virtualized server performance.

[0215] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are run in a computer, they are used to execute the above-mentioned method for establishing a virtualized server load response model.

[0216] A third aspect of the present invention provides a virtualized server load response model establishment system, which includes the above-mentioned computer-readable storage medium.

[0217] Specifically, the principle of the present invention is:

[0218] 1. Load Testing and Data Collection: First, the initial conditions for the virtualized server were set, including the physical server configuration, virtualization platform type, number of virtual machines, service type, network topology, and initial resource allocation strategy. Then, multiple load tests were conducted based on different initial conditions, recording metrics such as CPU utilization, memory usage, storage I / O, network throughput, response time, and energy consumption to construct a virtualized server load dataset.

[0219] 2. Establishing Load Response Characteristic Equations: Based on the collected load dataset, we established equations for computing resources, storage resources, network resources, service performance, energy consumption, and virtualization overhead. These equations describe the quantitative relationships between various performance indicators and key influencing factors such as the number of virtual CPU cores, the number of physical CPU cores, load intensity, number of virtual machines, read and write operation frequency, and the number of concurrent requests. Furthermore, we constructed a resource scheduling equation to reflect the resource allocation efficiency of the virtualized server.

[0220] 3. Resource Conversion and Competition Modeling: A resource conversion model is established for the conversion of physical resources to virtual resources, taking into account factors such as virtualization layer overhead and resource loss. Furthermore, a resource competition model is established based on the resource demand competition and interference relationships between virtual machines to describe the mutual impact of virtual machine performance.

[0221] 4. Solving the load response characteristics: Integrate the resource conversion model and resource competition model into the aforementioned load response characteristic equation group, and use numerical solution methods to obtain a complete numerical solution matrix, which describes the numerical characteristics of indicators such as CPU utilization, storage I / O, network throughput, response time, energy consumption, virtualization overhead and resource scheduling efficiency of the virtualization server under different load conditions.

[0222] 5. Load Response Model Construction: Finally, based on the numerical solution matrix, a virtualized server load response model was established, which accurately predicts server performance under varying load, resource, environmental, and configuration conditions. This model uses multiple linear regression analysis to determine the weight coefficients of various influencing factors, reflecting their relative contributions to server performance.

[0223] The core principle of the above technical solution is:

[0224] 1) Make full use of a large amount of measured data to establish a set of equations covering various key performance indicators of virtualized servers, providing a basis for subsequent modeling.

[0225] 2) By integrating the two major mechanisms of resource conversion and resource competition, it deeply depicts the mapping relationship from physical resources to virtual resources in the virtualized environment, as well as the mutual influence between virtual machines, and enhances the interpretability of the model.

[0226] 3) Using numerical solution technology, the performance characteristic numerical matrix under different load conditions is efficiently obtained to provide supporting data for the construction of the final load response model.

[0227] 4) The weight coefficients of various influencing factors were determined by using multiple linear regression analysis, which shows that the method of the present invention has good adaptability and predictive ability.

[0228] In summary, the virtualization server load response model establishment method proposed in this invention fully combines mathematical modeling and numerical analysis technology, and theoretically constructs a comprehensive, accurate and explainable virtualization performance analysis framework and model.

[0229] To better understand and implement the present invention, the following describes a specific application scenario: an internet company is deploying a new virtualized server cluster to support its web services, database, and big data analytics services. They urgently need to understand the load response characteristics of the servers in this virtualized environment to facilitate proper resource allocation and dynamic scheduling. The company has decided to employ the virtualized server load response modeling method proposed in this invention to conduct comprehensive performance analysis and prediction for this virtualized environment.

[0230] First, the company determined the initial virtualization server configuration. The physical server used an Intel Xeon Gold 6240R 24-core processor with a clock speed of 2.4 GHz, 256 GB of memory, and 12 TB of SAS hard drives. VMware vSphere 7.0 was used as the virtualization platform, initially deploying 20 virtual machines. Ten of these VMs ran web services, five ran MySQL databases, and five ran Hadoop big data analytics clusters. The network topology used 10 Gigabit Ethernet, with each VM allocated 4 vCPUs, 16 GB of memory, and 500 GB of virtual storage. VMware's DRS dynamic load balancing strategy was used for resource allocation.

[0231] Based on the above initial conditions, the enterprise began to load test and collect data on the virtualized environment.

[0232] 1. Load testing and data collection

[0233] For three different business types: Web services, databases, and big data analysis, the company designed corresponding load testing solutions:

[0234] (1) Web service load test: Use the Apache Bench tool to simulate concurrent user requests. The test setting increases the concurrency from 10 to 100, increasing by 10 each time. Record indicators such as CPU usage, memory usage, network throughput, and response time.

[0235] (2) Database load test: Use the sysbench tool to perform a stress test on the MySQL database, including select, insert, update, and delete operations. The test database table size increases from 100MB to 1GB, with each increase of 100MB. Record indicators such as CPU usage, memory usage, disk IOPS, and response time.

[0236] (3) Big Data Analysis Load Testing: Use the official test program to test MapReduce jobs on the Hadoop cluster. The test data size increases from 100GB to 1TB, with each increment of 100GB. Record indicators such as CPU usage, memory usage, disk throughput, and job execution time.

[0237] By adjusting the load intensity and data size multiple times, the company collected 30 sets of virtualized server performance test data under different conditions, forming the initial load data set D0. Table 1 below lists some of the data samples:

[0238] Table 1 Data sample table

[0239]

[0240]

[0241] These test data cover the performance of different business types and load intensities in a virtualized environment, laying the foundation for the subsequent establishment of a load response model.

[0242] 2. Establishment of load response characteristic equation

[0243] Based on the collected load test data set D0, the enterprise began to establish a set of load response characteristic equations for virtualized servers.

[0244] (1) Calculation resource equation:

[0245]

[0246] Among them, C u is the CPU usage, V cpu is the number of virtual CPU cores, P cpu is the number of physical CPU cores, L is the number of concurrent connections or data size, N is the number of virtual machines, α1, α2, and α3 are unknown coefficients, and ε1 is the error term. Through multiple linear regression analysis of the test data, we found that α1 = 0.6, α2 = 0.015, and α3 = 0.02.

[0247] (2) Storage resource equation:

[0248]

[0249] Among them, I o For storage IOPS, V storage is the virtual storage capacity, P storage is the physical storage capacity, R is the read operation frequency, W is the write operation frequency, β1, β2, β3 are unknown coefficients, and ε2 is the error term. Regression analysis yields β1 = 0.8, β2 = 100, and β3 = 150.

[0250] (3) Network resource equation:

[0251]

[0252] Among them, T p is the network throughput, V bandwidth is the virtual network bandwidth, P bandwidth is the physical network bandwidth, P s is the number of packets sent, P r is the number of received data packets, γ1, γ2, γ3 are unknown coefficients, and ε3 is the error term. Regression analysis yields γ1 = 0.9, γ2 = 0.002, and γ3 = 0.003.

[0253] (4) Business performance equation:

[0254] R t =δ1·C u +δ2·I o +δ3·T p +δ4·Q+ε4;

[0255] Among them, R t is the response time or job execution time, Q is the number of concurrent requests or data size, δ1, δ2, δ3, and δ4 are unknown coefficients, and ε4 is the error term. Regression analysis yields δ1 = 2.5, δ2 = 0.0005, δ3 = 0.001, and δ4 = 0.01.

[0256] (5) Energy consumption equation:

[0257] E c =η1·C u +η2·I o +η3·T p +η4·T+ε5;

[0258] Among them, E c is the energy consumption, T is the running time, η1, η2, η3, η4 are the unknown coefficients, and ε5 is the error term. Regression analysis shows that η1 = 30, η2 = 0.05, η3 = 0.1, and η4 = 10.

[0259] (6) Virtualization overhead equation:

[0260]

[0261] Among them, O v is the virtualization overhead, V total is the total virtual resource, P total is the total physical resource, S f is the resource switching frequency, θ1, θ2, θ3 are unknown coefficients, and ε6 is the error term. Regression analysis yields θ1 = 2, θ2 = 5, θ3 = 20.

[0262] (7) Resource scheduling equation:

[0263]

[0264] Among them, S e is the scheduling efficiency, A r R is the allocated resource. r To request resources, B r is the resource balance, M t is the number of migrations, λ1, λ2, λ3 are unknown coefficients, and ε7 is the error term. Regression analysis yields λ1 = 0.8, λ2 = 0.15, and λ3 = 0.05.

[0265] Through the above steps, the enterprise established a complete set of virtualization server load response characteristic equations, covering key indicators such as CPU, memory, storage, network, business performance, energy consumption and resource scheduling, laying the foundation for the subsequent establishment of resource conversion model and resource competition model.

[0266] 3. Resource conversion and resource competition modeling

[0267] (1) Resource conversion model

[0268] Based on the aforementioned configuration information of physical servers and virtual machines, the enterprise established the following resource conversion model:

[0269] V r =φ(P r ,H t ,V o )=κ1·P r ·(1-H t )-κ2·V o +ε8;

[0270] Among them, V r For virtual resources, P r For physical resources, H t is the virtualization layer overhead ratio, V o is the resource virtualization loss, κ1=0.9, κ2=0.1, and ε8 is the error term.

[0271] Through the analysis of measured data, the enterprise determined the virtualization layer overhead ratio of VMware vSphere 7.0 t About 10%, resource virtualization loss V o Therefore, the conversion efficiency of physical resources to virtual resources is about 81%.

[0272] (2) Resource competition model

[0273] In response to resource competition between virtual machines, the enterprise has established the following resource competition model:

[0274] C f =ψ(R d ,I l ,P c )=μ1·R d +μ2·I l +μ3·P c +ε9;

[0275] Among them, C f is the competitive factor, R d is the resource demand, I l is the interference level, P c is the priority coefficient, μ1=0.6, μ2=0.3, μ3=0.1, and ε9 is the error term.

[0276] After analysis, the company found that the web service VMs had the highest resource requirements, followed by the database VMs, and the big data analytics VMs had the lowest. Furthermore, the web service and database VMs experienced significant interference with each other, while the big data analytics VMs had less interference with other VMs. Furthermore, the company implemented a dynamic load balancing strategy called DRS, which resulted in relatively balanced priority levels across VMs.

[0277] 4. Numerical solution matrix construction

[0278] The resource conversion model and resource competition model were integrated into the aforementioned load response characteristic equations. The enterprise used the Newton-Raphson iteration method to numerically solve the equations, ultimately obtaining a 10×7 numerical solution matrix M, as shown in Table 2 below:

[0279] Table 2 Numerical solution matrix

[0280]

[0281]

[0282] The numerical solution matrix completely describes the performance of the virtualized environment under different load conditions, providing basic data for the subsequent establishment of a load response model.

[0283] 5. Load response model construction

[0284] Finally, the enterprise used the aforementioned numerical solution matrix M to establish the following virtualized server load response model:

[0285] P m =f(L,R,E,C)=ω1·L+ω2·R+ω3·E+ω4·C+ε 10 ;

[0286] Among them, P m is the performance index, L is the load vector, R is the resource vector, E is the environment vector, C is the configuration vector, ω1=0.5, ω2=0.3, ω3=0.15, ω4=0.05, ε 10 is the error term.

[0287] The load vector L includes the number of concurrent requests, data size, etc., the resource vector R includes CPU usage, storage IOPS, network throughput, etc., the environment vector E includes virtualization platform type, network topology, etc., and the configuration vector C includes the number of virtual machines, resource allocation strategy, etc.

[0288] Through multiple linear regression analysis, the company determined the weight coefficients above, which reflect the relative impact of different factors on virtualized server performance. Among them, load intensity is the most important influencing factor, followed by resource usage, and environmental and configuration factors are relatively small.

[0289] To verify the accuracy of the load response model, the company selected 10% of the test data as an independent validation set. The results showed that the model's prediction root mean square error was only 8%, which was able to better meet the performance prediction requirements of the virtualized environment. The specific test chart is as follows: Figure 2-Figure 6 shown.

[0290] Figure 2 Resource usage comparison of different workloads. This is a grouped bar chart showing the comparison of CPU usage, memory usage, storage IOPS, and network throughput for three workloads: web services, databases, and big data analytics.

[0291] Figure 3 3D graph showing the relationship between load percentage, concurrency, and performance. This graph is a three-dimensional surface graph that shows the relationship between load percentage, concurrency, and performance metrics. The color gradient reflects performance changes.

[0292] Figure 4 Virtual Machine Resource Allocation Heat Map. This heat map shows how different types of virtual machines (Web, Database, Hadoop) allocate CPU, memory, storage, and network resources. Darker colors indicate greater allocation.

[0293] Figure 5 Performance Metric Time Series: This is a dual-axis time series chart. The top half shows CPU usage over time, and the bottom half shows response time over time. This helps analyze the temporal patterns and correlations of performance metrics.

[0294] Figure 6 This diagram shows the resource competition between different virtual machines. Node size indicates the resource requirements of the virtual machine, color indicates the virtual machine type, and edge thickness indicates the intensity of competition.

[0295] Based on the constructed virtualized server load response model, enterprises can conduct comprehensive performance analysis and optimization of the virtualized cluster:

[0296] 1) For different business types, predict their resource consumption and performance under high load conditions to provide a basis for dynamic resource scheduling.

[0297] 2) Analyze the degree of resource competition between virtual machines, identify potential performance bottlenecks, and adjust resource allocation strategies to improve overall efficiency.

[0298] 3) Evaluate the impact of virtualization platform upgrades or physical resource expansion on performance, and provide support for IT infrastructure optimization decisions.

[0299] 4) Adjust resource allocation in advance based on business peak forecasts to ensure the achievement of service quality goals.

[0300] The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for establishing a virtualized server load response model, characterized in that: The following steps are involved: S10. Setting initial test conditions, performing load testing on the virtualized server, and obtaining test data; S20, changing the test conditions multiple times, performing multiple load tests, obtaining test data under different conditions, and merging each set of test conditions and corresponding test data into a virtualized server load data set; S30, establishing a load response characteristic equation group including all parameters in the virtualized server load data set; S40, fitting the virtualized server load data set to the load response characteristic equation group to obtain a fitted equation group; S50: Establish a resource conversion model based on the resource mapping relationship between the physical server and the virtual server ,in, For virtual resources, For physical resources, is the virtualization layer overhead ratio, For resource virtualization loss, and is the conversion factor, is the error term; S60. Establish a resource competition model based on the resource competition and interference relationship between multiple virtual servers within the same physical server. ,in is the competitive factor, is the resource demand, is the interference level, is the priority coefficient, is the weight coefficient, is the error term; S70, the resource conversion model and resource competition model is embedded into each load response characteristic equation, the modified set of equations is solved to obtain a numerical solution matrix of the virtualized server load response characteristics; S80. Based on the numerical solution matrix, a virtualized server load response model is constructed to describe the resource usage and performance of the virtual server under different load conditions.

2. The method for establishing a virtualized server load response model according to claim 1, wherein: The test conditions include physical server configuration, virtualization platform type, number of virtual machines, business type, network topology and initial resource allocation.

3. The method for establishing a virtualized server load response model according to claim 2, wherein: The test data includes CPU usage, memory usage, storage I / O, network throughput, response time and energy consumption.

4. The method for establishing a virtualized server load response model according to claim 3, wherein: The load response characteristic equation group includes computing resource equations, storage resource equations, network resource equations, service performance equations, energy consumption equations, virtualization overhead equations, and resource scheduling equations.

5. The method for establishing a virtualized server load response model according to claim 4, wherein: The method for fitting the virtualized server load data set to the load response characteristic equation group is the least squares method.

6. A method for establishing a virtualized server load response model according to claim 5, characterized in that: The resource conversion model is used to describe the conversion process from physical resources to virtual resources and its efficiency.

7. The method for establishing a virtualized server load response model according to claim 6, wherein: The resource competition model is used to describe the mutual influence between virtual servers.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, and when the program instructions are executed in a computer, they are used to execute the method for establishing a virtualized server load response model according to any one of claims 1 to 7.

9. A virtualized server load response model establishment system, characterized in that: Contains the computer-readable storage medium of claim 8.

Images