Cloud resource operation analysis method and device and computer equipment

Abstract

The invention relates to a cloud resource operation analysis method and device and computer equipment, and the method comprises the steps: collecting load data of various services at a plurality of historical moments according to a preset period, and determining plan factors at the historical moments according to the load data of various services at the plurality of historical moments; inputting the load data and the plan factors of various businesses at the historical moment into a resource use prediction model to obtain a resource use prediction result at the future moment output by the model; determining a constraint condition according to the resource use prediction result, and constructing a target function related to resource configuration and input cost; optimizing the objective function by using the constraint condition to obtain a load balancing optimization decision; and adjusting and configuring resources of various services according to the load balancing optimization decision. According to the method, resources are uniformly distributed among a plurality of servers or nodes, efficient operation of services and full utilization of the resources are ensured, the resource use cost is reduced, and the requirements of response time, delay and reliability of the services are met.

Description

Technical Field

[0001] This specification relates to the field of cloud resource operation technology, and in particular to a cloud resource operation analysis method, device and computer equipment. Background Art

[0002] The existing cloud computing resource management system realizes efficient resource utilization and optimization through centralized control and allocation of cloud resources. Cloud computing resource management systems usually include modules such as resource monitoring, load balancing, automatic expansion and billing. However, these systems have certain limitations in handling operational analysis and decision-making of large-scale cloud resources, including: when dealing with real-time monitoring and analysis of large-scale cloud resources, existing systems often have delays, cannot reflect resource usage in a timely manner, and lack real-time performance; due to the complex and changeable resource usage, it is difficult for existing systems to provide accurate resource usage analysis and prediction; most existing systems rely on preset rules for resource scheduling and optimization, lack adaptive and intelligent decision-making capabilities, and are difficult to meet dynamically changing resource needs, and have a low level of intelligence. Summary of the invention

[0003] In order to solve the problem that the existing technology cannot reflect the resource usage in time and there is a delay in real-time monitoring and analysis of large-scale cloud resources, the embodiments of this specification provide a cloud resource operation analysis method, device and computer equipment.

[0004] The embodiment of this specification provides a cloud resource operation analysis method, which includes: collecting load data of various types of services at multiple historical moments according to a preset period, and determining the planning factors at the historical moments according to the load data of various types of services at the multiple historical moments; inputting the load data of various types of services at historical moments and the planning factors into a resource usage prediction model to obtain resource usage prediction results for future moments output by the model; wherein the resource usage prediction model is obtained by training an LSTM initial model based on load sample data and sample planning factors of various types of services at multiple historical sample moments; determining constraints based on the resource usage prediction results, and constructing an objective function related to resource allocation and input cost; optimizing the objective function using the constraints to obtain a load balancing optimization decision; and adjusting and configuring resources for various types of services based on the load balancing optimization decision.

[0005] According to one aspect of an embodiment of the present specification, a resource usage prediction model is constructed in the following manner: a training sample data set is determined, the training sample data set including: load sample data, sample planning factors and resource usage sample results of various types of services at multiple historical sample moments, the load sample data including: payment sample data, photovoltaic usage sample data and charging pile usage sample data; according to the year and month in which the multiple historical sample moments are located, the planning factor corresponding to each historical sample moment is determined; the load sample data and the sample planning factor are input into the LSTM initial model to obtain the resource usage prediction sample result of the next sample moment output by the LSTM initial model; according to the resource usage sample result and the resource usage prediction sample result, the LSTM initial model is iteratively trained until the resource usage prediction sample result and the loss of the resource usage sample result are within a preset range, and it is determined that the resource usage prediction model is constructed.

[0006] According to one aspect of an embodiment of the present specification, a sample planning factor at a historical sample moment is determined in the following manner: a first increment of resource usage at a first historical sample moment is determined based on resource usage data at multiple moments before the first historical sample moment; a second increment of resource usage at a second historical sample moment is determined based on resource usage data at multiple moments before the second historical sample moment; a linear fit is performed on the first increment and the second increment to determine an increment of the historical sample moment compared to the historical sample moment compared to multiple moments before the historical sample moment; and a planning factor for the historical sample moment is determined based on the increment.

[0007] According to one aspect of an embodiment of this specification, constructing an objective function related to resource allocation and input cost includes: determining the product of the resource allocation amount and the resource unit cost of each type of resource; summing the products of each type of resource to obtain an objective function, and the objective function is as follows:

[0008] Total Cost represents the total cost of resource allocation, T represents the total time required to predict and allocate resources in the future, and C i represents the resource unit cost of the i-th resource, x i (t) represents the amount of resources allocated at each time t, and N represents the number of types of configured resources.

[0009] According to one aspect of an embodiment of this specification, optimizing the objective function using the constraint condition to obtain a load balancing optimization decision includes: using the following formula to constrain the objective function and determine the load balancing optimization decision: The constraints are: x i (t) represents the amount of resources allocated at each time t, N represents the number of types of configured resources; Ri (t) represents the predicted resource demand.

[0010] According to one aspect of the embodiments of this specification, a penalty term is introduced to optimize the objective function and determine a load balancing optimization decision.

[0011] According to one aspect of an embodiment of the present specification, the method further includes: adjusting and configuring resources for various types of services based on the load balancing optimization decision, including: determining the number of service nodes deployed in the cloud platform based on the proportion of various types of resources corresponding to each business in the load balancing optimization decision.

[0012] The embodiment of the present specification also provides a cloud resource operation analysis, and the device includes: a collection unit, which is used to collect load data of various types of services at multiple historical moments according to a preset period, and determine the planning factors of the historical moments according to the load data of various types of services at the multiple historical moments; a prediction result determination unit, which is used to input the load data of various types of services at historical moments and the planning factors into a resource usage prediction model to obtain resource usage prediction results for future moments output by the model; wherein the resource usage prediction model is obtained by training an LSTM initial model based on load sample data and sample planning factors of various types of services at multiple historical sample moments; an objective function construction unit, which is used to determine constraints based on the resource usage prediction results, and construct an objective function related to resource allocation and input costs; an optimization decision determination unit, which is used to optimize the objective function using the constraints to obtain a load balancing optimization decision; a resource configuration unit, which is used to adjust and configure resources for various types of services based on the load balancing optimization decision.

[0013] The embodiment of the present specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the cloud resource operation analysis method is implemented when the processor executes the computer program.

[0014] The embodiments of the present specification also provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cloud resource operation analysis method is implemented.

[0015] This manual realizes real-time monitoring and analysis of cloud resource usage data through distributed data collection and processing, significantly improving the system's response speed and decision-making timeliness. The use of deep learning and big data analysis technology can accurately predict resource usage trends, provide more accurate optimization decisions, reduce resource waste and performance bottlenecks. Improve the flexibility and adaptability of the system in responding to dynamically changing resource demands. Significant technical progress has been made in real-time, accuracy and intelligence, which can effectively solve key problems in existing technologies and provide efficient cloud resource management solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of a cloud resource operation analysis method according to an embodiment of this specification;

[0018] Figure 2 The figure is a flow chart of a method for constructing a resource usage prediction model according to an embodiment of the present specification;

[0019] Figure 3 The figure is a flow chart of a method for determining a sample planning factor at a historical sample moment according to an embodiment of the present specification;

[0020] Figure 4 The figure is a flow chart of a method for constructing an objective function according to an embodiment of the present specification;

[0021] Figure 5 The figure is a flow chart of a method for configuring service resources according to an embodiment of this specification;

[0022] Figure 6 This is a schematic diagram of the structure of a cloud resource operation analysis device according to an embodiment of this specification;

[0023] Figure 7 Shown is a schematic diagram of a computer device provided in an embodiment of this specification.

[0024] Description of the accompanying symbols:

[0025] 601, collection unit;

[0026] 602. prediction result determination unit;

[0027] 603, objective function construction unit;

[0028] 604. Optimizing decision-making unit;

[0029] 605. Resource allocation unit;

[0030] 702. Computer equipment;

[0031] 704, processor;

[0032] 706. Memory;

[0033] 708, driving mechanism;

[0034] 710, input / output module;

[0035] 712. Input devices;

[0036] 714. Output devices;

[0037] 716. Presentation equipment;

[0038] 718. Graphical user interface;

[0039] 720, network interface;

[0040] 722. Communication link;

[0041] 724. Communication bus. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0043] It should be noted that the terms "first", "second", etc. in the description and claims of this specification and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0044] This specification provides method operation steps as described in the embodiments or flow charts, but more or fewer operation steps may be included based on routine or non-creative work. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the system or device product is executed in practice, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel.

[0045] It should be noted that the cloud resource operation analysis method, device and computer equipment in this specification can be used in the field of cloud resource operation technology, and can also be used in other fields. This specification does not limit the application field of the cloud resource operation analysis method, device and computer equipment.

[0046] Figure 1 The flowchart of a cloud resource operation analysis method according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0047] Step 101, collecting load data of various services at multiple historical moments according to a preset cycle, and determining a planning factor at the historical moment according to the load data of various services at the multiple historical moments.

[0048] In this step, various types of services include but are not limited to: one or any combination of different types of services such as marketing payment, photovoltaic energy use and charging pile use. Correspondingly, in order to predict resource use at a certain moment or a certain period in the future, it is necessary to count the load data of various types of services at historical moments. The load data of various types of services at historical moments include but are not limited to: how many users visited or how many transactions were made at the historical moment.

[0049] In the embodiments of this specification, historical moments refer to any moments relative to future moments or moments before they have not yet arrived. Specifically, the load data related to various types of businesses in historical moments in cloud resources are collected through API interfaces or SDKs. The load data of various types of businesses are related to marketing payment, photovoltaic use, and charging pile use. The specific load data include, but are not limited to: CPU usage, memory occupancy, storage usage, network bandwidth utilization, the number of concurrent processes of the program, etc. Specifically, according to a preset cycle, multiple historical moments are collected, such as marketing payment, photovoltaic use, and charging pile use. Different types of businesses have how many user visits or transaction volumes a certain business has on a certain day or at a certain moment. Furthermore, the planning factor of the historical moment is determined based on the load data of various types of businesses at the multiple historical moments. In the embodiments of this specification, the planning factor is used as a training sample data set to be input into the LSTM initial model to adjust the prediction results output by the resource usage prediction model.

[0050] This specification embodiment shows a code example for timing collection of load data:

[0051]

[0052] This step also includes cleaning, organizing and storing the collected data, including: removing invalid or abnormal data to ensure data quality; standardizing data and unifying data formats to facilitate subsequent analysis; clustering data and trend prediction to identify different resource usage patterns.

[0053] The embodiments of this specification provide code examples for data cleaning and data preprocessing:

[0054] The Pandas library is used to clean the collected data, including denoising, normalization, and processing of missing values.

[0055] Data storage: Use a distributed database such as MongoDB or Hadoop HDFS to store preprocessed data for subsequent analysis.

[0056] Sample code:

[0057]

[0058] Step 102, input the load data of various types of business at historical moments and the planning factors into the resource usage prediction model to obtain the resource usage prediction results at future moments output by the model; wherein the resource usage prediction model is obtained by training the LSTM initial model based on the load sample data and sample planning factors of various types of business at multiple historical sample moments.

[0059] In this step, the load data of various services at multiple historical moments collected in step 101 and the corresponding planning factors are input into the resource usage prediction model to obtain the resource usage prediction results for future moments output by the resource usage prediction model, that is, the prediction results of different types of resources such as how much CPU usage, how much memory usage, or how much network bandwidth utilization should be allocated to a certain service. Specifically, the resource usage prediction results for future moments output by the model include, but are not limited to: one of CPU usage, memory usage, storage usage, network bandwidth utilization, the number of concurrent processes of a program, etc., or any combination thereof.

[0060] In the embodiment of this specification, the resource usage prediction model is obtained by training the LSTM initial model based on the load sample data, sample planning factors and load sample data of various services at multiple historical sample moments. Figure 2 describe.

[0061] This specification embodiment shows an example code for predicting future resource requirements using a resource prediction model:

[0062] predicted_resource=model.predict(X[1].reshape(1,10,1))

[0063] print(predicted_resource)

[0064] Step 103, determining constraint conditions according to the resource usage prediction result, and constructing an objective function related to resource allocation and input cost.

[0065] In the embodiment of this specification, in order to ensure that resource allocation meets the predicted demand, the constraint condition can be defined as follows: at each time t, the amount of allocated resources x i (t) At least meet the predicted resource demand R i (t), that is: This constraint ensures that the system does not allocate less than the predicted resource requirements. i (t) is the resource usage prediction result determined in step 102.

[0066] Furthermore, in order to ensure that the business resources allocated for a certain time in the future can meet the business load requirements at that time in the future, an objective function is further constructed. In this step, the objective function can be determined by multiplying the amount of allocated resources by the unit cost of resources and summing all resources. Among them, the objective function is related to resource allocation and input cost, and it is expected that the cost of resource allocation is minimized and resource allocation meets the predicted requirements. The cost of resource allocation can be expressed by multiplying the amount of allocated resources by the unit cost of resources and summing all resources. For a detailed description of the objective function, see Figure 4 , this application will not be elaborated here.

[0067] Step 104: Utilize the constraint conditions to optimize the objective function and obtain a load balancing optimization decision.

[0068] The objective function is optimized using constraints so that the final objective function minimizes the total resource usage cost while ensuring that resource requirements are met.

[0069] Step 105: Adjust and configure resources for various services according to the load balancing optimization decision.

[0070] Through the optimized objective function, the resources allocated to each business are determined to achieve load balancing. The load balancing optimization decision outputs the allocation results of class resources in each time period, and the load balancing plan for each business module, including task allocation, node migration and other operations. Furthermore, the optimization results of system resource usage cost and business performance evaluation are obtained. Ensure the balance of resource usage to avoid the situation where some node resources are overloaded while other node resources are idle.

[0071] This manual evenly distributes resources among multiple servers or nodes to ensure efficient operation of the business and full utilization of resources. In load balancing optimization, the balance between resource cost and system performance must be considered at the same time. By optimizing the allocation of resources, the cost of resource use can be minimized. Ensure that resource allocation meets the response time, latency and reliability requirements of the business, and ensure that system performance is not affected.

[0072] Figure 2 The flowchart of a method for constructing a resource usage prediction model according to an embodiment of the present specification specifically includes the following steps:

[0073] Step 201, determine a training sample data set, the training sample data set includes: load sample data, sample planning factors and resource usage sample results of various types of services at multiple historical sample moments, the load sample data includes: payment sample data, photovoltaic usage sample data and charging pile usage sample data.

[0074] In this step, the payment sample data, photovoltaic usage sample data and charging pile usage sample data at multiple historical sample moments are used as training sample data sets. The sample planning factors corresponding to the payment sample data, photovoltaic usage sample data and charging pile usage sample data at multiple historical sample moments are used as label data for subsequent model training.

[0075] For example, the resource usage sample results corresponding to the load sample data of the marketing payment system at a certain historical sample time are: the CPU occupancy rate of the marketing payment system at time T is N%, the memory occupancy rate is M%, the storage occupancy is P, and the network resource occupancy is W.

[0076] Step 202, determining the planning factor corresponding to each historical sample moment according to the years and months in which the multiple historical sample moments are located.

[0077] In this step, each historical sample moment has a corresponding planning factor. The planning factor is used to adjust the prediction results output by the resource usage prediction model. In some embodiments of this specification, the planning factor is related to the historical or current economic level, consumption level or policy factors. The planning factor can be expressed as: the business volume at a certain moment in the current year shows an increasing trend compared with the business volume at the same period / time in previous years.

[0078] Step 203: input the load sample data and the sample planning factor into the LSTM initial model to obtain the resource usage prediction sample result at the next sample moment output by the LSTM initial model.

[0079] In this step, the LSTM initial model can be a long short-term memory network model, including a forget gate, an input gate, and an output gate. The load sample data and sample planning factors of various types of services at multiple historical sample moments in the training sample data set are converted into vector or matrix form and input into the LSTM initial model. For each time step t, the output of the forget gate, input gate, and output gate are calculated. According to the output of the forget gate and the cell state of the previous time step, the cell state of the current time step is calculated. According to the output of the input gate and the candidate value of the current time step, the cell state is updated. According to the output of the output gate and the cell state of the current time step, the hidden state of the current time step is calculated.

[0080] In some embodiments of the present specification, in order to predict the resource usage prediction results at future moments (such as t0+n, etc.), an input time series with a step size of n is input into the LSTM initial model. The resource usage prediction data at t0+1 can be quickly and accurately obtained through network forward propagation. After completing a prediction, the input sequence window will move back by one step size. Thus, the resource usage prediction data at t0+2 is predicted to achieve the prediction of resource usage at multiple moments in the future.

[0081] Step 204, iteratively train the LSTM initial model according to the resource usage sample results and the resource usage prediction sample results, until the loss of the resource usage prediction sample results and the resource usage sample results is within a preset range, and determine that the resource usage prediction model is built.

[0082] In this step, the loss function value is calculated based on the output prediction results and true labels of the LSTM initial model. Common loss functions include cross entropy loss, etc. Back propagation is performed based on the loss to update the network parameters in the LSTM initial model. The LSTM initial model is iterated multiple times until the loss between the prediction results output by the LSTM model and the resource usage sample results meets the preset convergence value, or the number of iterations of the LSTM initial model reaches the preset value, and a trained resource usage prediction model is obtained.

[0083] In this step, the gradient of the loss function with respect to each parameter can be calculated by the chain rule. Repeat the process of forward propagation, loss calculation, and back propagation until the model's performance on the validation set reaches the best or the number of training rounds reaches the preset value.

[0084] This specification also shows the sample code of the training model:

[0085] Sample code:

[0086]

[0087]

[0088] Figure 3 The flowchart of a method for determining a sample planning factor at a historical sample moment in an embodiment of the present specification is shown, which specifically includes the following steps:

[0089] Step 301, determine a first increment of resource usage at the first historical sample moment based on resource usage data at multiple moments before the first historical sample moment. In this step, multiple moments before the first historical sample moment represent at least one moment before the first historical sample moment. The first historical sample moment can also be understood as a period in the past, and multiple moments before the first historical sample moment can also be understood as at least one period in the past. The first increment of resource usage at the first historical sample moment is the increase or decrease in resource usage data at the first historical sample moment compared to the resource usage data at multiple moments before the first historical sample moment.

[0090] For example, assuming that the first historical sample moment is November 2022, the multiple moments before the first historical sample moment may be August 2022 and September 2022, and the first increment is the increment of resource usage in November 2022 compared with the resource usage in August and September. For another example, assuming that the first historical sample moment is October 1, 2022, the multiple moments before the first historical sample moment may be September 1, 2022 and August 1, 2022, and the first increment is the increment of resource usage on October 1, 2022 compared with the resource usage on August 1, 2022 and September 1, 2022. Assume that the first increment is 0.2%.

[0091] Step 302: Determine a second increment of resource usage at a second historical sample moment based on resource usage data at multiple moments before the second historical sample moment. Assume that the second increment is 0.3%.

[0092] The second historical sample moment in this step is different from the first historical sample moment in step 301. The first historical sample moment and the second historical sample moment can be moments or time periods in different years but the same month, moments or time periods in the same year but different months, moments or time periods in different weeks of the same month, moments or time periods in the same week but different days, or moments or time periods in the same day but different time periods. For example, the second historical sample moment is November 2021, and the multiple moments before the second historical sample moment can be October 2021 and September 2021, or September 2021 and August 2021. This application does not limit the specific time of the first historical sample moment, the second historical sample moment, the multiple moments before the first historical sample moment, and the multiple moments before the second historical sample moment. Correspondingly, the second increment of resource usage at the second historical sample moment in this step is the increment of resource usage in November 2021 compared with October and September 2021, or September and August.

[0093] Step 303, linearly fit the first increment and the second increment to determine the increment of the historical sample moment compared to the multiple moments before the historical sample moment. In this step, after linearly fitting the first increment and the second increment, a fitting result is obtained; the increment of the historical sample moment compared to the multiple moments before the historical sample moment is determined based on the fitting result. In the embodiment of this specification, it can be considered that: the increments of the first historical sample moment and the second historical sample moment occurring before the historical sample moment compared to the multiple moments before them have certain trends and regularities, and the increment of the historical sample moment compared to the multiple moments before it can be determined based on such trends and regularities.

[0094] According to the embodiments described above, based on the increment of resource usage in November 2021 compared to October and September 2021, or September and August 2021, and the increment of resource usage in November 2022 compared to September and August 2022, the increment of historical sample moment (for example, November 2023) compared to September and August 2023 can be determined.

[0095] Step 304: determine the planning factor at the historical sample moment according to the increment.

[0096] According to the embodiment in step 303, after determining the increment of the historical sample moment compared to multiple moments before the historical sample moment, the planning factor of the historical sample moment can be determined according to the increment. Based on the embodiment described above, the first increment is 0.2%, and the value of the second increment is 0.3%. The first increment and the second increment are linearly fitted, and it can be estimated that the planning factor of the original historical moment is 0.4%. The calculated planning factor and the load sample data of various types of business at multiple historical sample moments are used as a training sample data set for input into the LSTM initial model to obtain the prediction result output by the resource usage prediction model.

[0097] Figure 4 The flowchart of a method for constructing an objective function according to an embodiment of the present specification is shown, which specifically includes the following steps:

[0098] Step 401, determine the product of the resource allocation amount and the resource unit cost of each type of resource.

[0099] In this step, after the resource usage prediction model is used to output the resource usage prediction results at the future time, various types of resources are allocated to various types of businesses according to the resource usage prediction results. The resource unit cost of each type of resource is a preset value and is a known quantity. The resource allocation amount of each type of resource is determined based on the resource usage prediction results. Resource types include computing resources, storage resources, etc.

[0100] Step 402, summing the products of various types of resources to obtain an objective function.

[0101] The objective function is as follows: Among them, Total Cost represents the objective function, and also represents the total cost of configuring various resources. T represents the total time required to predict and configure resources in the future, which can be one hour or one day. i represents the resource unit cost of the i-th resource, x i (t) represents the resource allocation amount of the i-th resource allocated at each time t, and N represents the number of types of configured resources. After constructing the objective function, this specification expects that the resource allocation cost is minimized.

[0102] In the embodiment of this specification, the constraint conditions are further used to optimize the objective function to obtain a load balancing optimization strategy. Specifically, it includes:

[0103] Combining the goal of minimizing resource cost and the constraints related to resource demand, the objective function can be formulated as the following constrained optimization problem:

[0104] Among them, T represents the total time required to predict and allocate resources in the future, C i represents the resource unit cost of the i-th resource, x i (t) represents the amount of resources allocated at each time t, and N represents the number of types of configured resources.

[0105] The constraints are:

[0106]

[0107] Based on this constraint, the objective function is converted into a load balancing optimization decision to minimize the total cost of resource usage while meeting resource requirements.

[0108] By optimizing the objective function, the system will dynamically adjust resource allocation according to different business load requirements; the optimization process will ensure that at each time t, all types of resources x i The distribution of (t) satisfies the demand forecast R i (t), while minimizing the total cost; the system dynamically adjusts resource allocation according to real-time business needs to ensure that each type of business load can get sufficient resource support; dynamic allocation according to different business needs ensures the rational use of resources.

[0109] In the embodiments of this specification, in order to further improve resource utilization, a penalty term is introduced to optimize the objective function and determine the load balancing optimization decision. This is to encourage resource allocation not to exceed demand too much. Specifically, according to the following formula

[0110]

[0111] Among them, λ is a weight parameter used to control the penalty for excess resource allocation. According to the penalty formula, the objective function constructed above is further constrained to control resource allocation within a reasonable range.

[0112] By optimizing the objective function according to the constraints and penalty formula, corresponding load balancing optimization decisions can be obtained, so that the use of resources achieves the best economic benefits while meeting business needs.

[0113] In the embodiments of this specification, the load balancing optimization decision also requires resource scheduling between the edge device and the cloud server. On the edge computing device, business requests are processed in real time to reduce the load pressure on the cloud server; the edge device will dynamically allocate computing and storage resources according to the real-time load. When the edge device cannot meet the business needs, the business request is transmitted to the cloud server, and the cloud allocates more resources through the optimization model to ensure that the task is completed on time.

[0114] In an embodiment of the present specification, adjusting and configuring resources of various types of services according to the load balancing optimization decision includes: determining the number of service nodes deployed in the cloud platform according to the proportion of various types of resources corresponding to each business in the load balancing optimization decision.

[0115] Specifically, container orchestration tools such as Kubernetes are used in conjunction with optimization decisions to automatically adjust the allocation of cloud resources, for example, increasing or decreasing computing nodes.

[0116] The following is the sample code:

[0117]

[0118] If each node accounts for 0.2%, if the predicted demand is higher than 80%, the number of nodes needs to be increased.

[0119] In the embodiment of this specification, determining the number of service nodes deployed in the cloud platform according to the proportion of various resources corresponding to each business in the load balancing optimization decision includes:

[0120] According to the various resources allocated to various services in the load balancing optimization strategy and the preset ratio of various resources occupied by each node, it is determined whether there are remaining resources after allocating resources based on the current number of nodes;

[0121] If there are remaining resources, determining whether a preset ratio of the remaining resources to the various types of resources occupied by the node exceeds a preset threshold;

[0122] If exceeded, a new node is allocated;

[0123] If not exceeded, no new node is allocated.

[0124] For example, the CPU resources allocated to the marketing payment service in the load balancing optimization strategy are 0.5%, and there are currently two nodes deployed in the marketing payment service, and the preset ratio of CPU resources occupied by each node is 0.2%. Then, there are 0.1% of resources allocated to the marketing payment service in the load balancing optimization strategy that have not been allocated, that is, the allocated CPU remaining resources are 0.1%. Then it is further determined whether the proportion of unallocated resources / remaining resources to the CPU resources occupied by each node exceeds 80%. If so, a new node is allocated to the service; if not, no new node is allocated. Obviously, the remaining resources of 0.1% do not exceed 80% of the preset ratio of 0.2%, so there is no need to allocate new resources to the marketing payment service; if it exceeds 80%, new resources are allocated to the marketing payment service, thereby realizing the effect of dynamically adjusting resource allocation according to real-time business needs and ensuring that each type of business load can get sufficient resource support.

[0125] Figure 5 The flowchart of a method for configuring service resources in an embodiment of the present specification is shown, which specifically includes the following steps:

[0126] Step 501, determining the service level according to the preset importance of the service type.

[0127] Step 502: configure resources corresponding to the service according to the service level and load balancing optimization strategy.

[0128] In this specification, each business type has a corresponding preset importance, and the business level can be determined according to the preset importance. For example, the charging pile business, as an emerging industry, belongs to a relatively important business type, so the charging pile business is determined to be a level one business type; marketing payment belongs to a regular business type, so the marketing payment business is determined to be a level three business level, and the level one is higher than the level three. This specification does not limit the corresponding relationship between the level size and the business importance.

[0129] Furthermore, according to the load balancing optimization strategy described above, combined with the service level determined here, the optimal configuration of resources is achieved for each type of service. For example, based on the load balancing optimization strategy corresponding to the charging pile service at the first service level, more nodes are allocated to the charging pile service.

[0130] Figure 6 This is a schematic diagram of the structure of a cloud resource operation analysis device according to an embodiment of this specification. In this figure, the basic structure of the cloud resource operation analysis device is described. The functional units and modules therein can be implemented in software, or a general chip or a specific chip can be used to implement cloud resource operation analysis. The device specifically includes:

[0131] The collection unit 601 is used to collect the load data of various services at multiple historical moments according to a preset period, and determine the planning factor of the historical moment according to the load data of various services at the multiple historical moments;

[0132] The prediction result determination unit 602 is used to input the load data of various services at historical moments and the planning factors into the resource usage prediction model to obtain the resource usage prediction results at future moments output by the model; wherein the resource usage prediction model is obtained by training the LSTM initial model based on the load sample data and sample planning factors of various services at multiple historical sample moments;

[0133] An objective function construction unit 603 is used to determine constraint conditions according to the resource usage prediction result and construct an objective function related to resource allocation and input cost;

[0134] The optimization decision determination unit 604 is used to optimize the objective function by using the constraint conditions to obtain a load balancing optimization decision;

[0135] The resource configuration unit 605 is used to adjust and configure resources of various services according to the load balancing optimization decision.

[0136] This manual evenly distributes resources among multiple servers or nodes to ensure efficient operation of the business and full utilization of resources. In load balancing optimization, the balance between resource cost and system performance must be considered at the same time. By optimizing the allocation of resources, the cost of resource use can be minimized. Ensure that resource allocation meets the response time, latency and reliability requirements of the business, and ensure that system performance is not affected.

[0137] like Figure 7The diagram shows a schematic diagram of a computer device provided by an embodiment of the present specification. The cloud resource operation analysis method described in the present application can be applied to the computer device. The computer device 702 may include one or more processors 704, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. The computer device 702 may also include any memory 706, which is used to store any kind of information such as code, settings, data, etc. Non-limitingly, for example, the memory 706 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 702. In one case, when the processor 704 executes an associated instruction stored in any memory or a combination of memories, the computer device 702 may perform any operation of the associated instruction. The computer device 702 also includes one or more drive mechanisms 708 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0138] The computer device 702 may also include an input / output module 710 (I / O) for receiving various inputs (via input devices 712) and for providing various outputs (via output devices 714). A specific output mechanism may include a presentation device 716 and an associated graphical user interface (GUI) 718. In other embodiments, the input / output module 710 (I / O), the input device 712, and the output device 714 may not be included, and the computer device 702 may be used as a computer device in a network. The computer device 702 may also include one or more network interfaces 720 for exchanging data with other devices via one or more communication links 722. One or more communication buses 724 couple the components described above together.

[0139] The communication link 722 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 722 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0140] Corresponds to Figures 1 to 5 The method in the embodiment of the present specification also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are executed.

[0141] The embodiment of the present specification also provides a computer-readable instruction, wherein when the processor executes the instruction, the program therein causes the processor to execute the following Figures 1 to 5 The method shown.

[0142] It should be understood that in the various embodiments of this specification, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.

[0143] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this specification generally indicates that the associated objects before and after are in an "or" relationship.

[0144] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0146] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0147] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this specification.

[0148] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0149] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0150] Specific embodiments are used in this specification to illustrate the principles and implementation methods of this specification. The description of the above embodiments is only used to help understand the methods and core ideas of this specification. At the same time, for those skilled in the art, according to the ideas of this specification, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this specification.

Claims

1. A cloud resource operation analysis method, characterized in that: The method comprises: Collecting load data of various services at multiple historical moments according to a preset period, and determining a planning factor at the historical moment according to the load data of various services at the multiple historical moments; The load data of various services at historical moments and the planning factors are input into the resource usage prediction model to obtain the resource usage prediction results at future moments output by the model; wherein the resource usage prediction model is obtained by training the LSTM initial model based on the load sample data and sample planning factors of various services at multiple historical sample moments; Determining constraints based on the resource usage prediction results and constructing an objective function related to resource allocation and input costs; Using the constraints, the objective function is optimized to obtain a load balancing optimization decision; According to the load balancing optimization decision, resources for various types of services are adjusted and configured.

2. The method according to claim 1, characterized in that The resource usage prediction model is constructed as follows: Determine a training sample data set, the training sample data set including: load sample data, sample planning factors and resource usage sample results of various types of services at multiple historical sample moments, the load sample data including: payment sample data, photovoltaic usage sample data and charging pile usage sample data; Determine the planning factor corresponding to each historical sample moment according to the years and months in which the multiple historical sample moments are located; Input the load sample data and the sample planning factor into the LSTM initial model to obtain the resource usage prediction sample result at the next sample moment output by the LSTM initial model; According to the resource usage sample results and the resource usage prediction sample results, the LSTM initial model is iteratively trained until the loss of the resource usage prediction sample results and the resource usage sample results is within a preset range, and it is determined that the resource usage prediction model is constructed.

3. The method according to claim 2, characterized in that The sample planning factor at the historical sample moment is determined as follows: Determine a first increment of resource usage at the first historical sample moment according to resource usage data at multiple moments before the first historical sample moment; Determine a second increment of resource usage at the second historical sample moment according to resource usage data at a plurality of moments before the second historical sample moment; Performing linear fitting on the first increment and the second increment to determine the increment of the historical sample moment compared to the historical sample moment and compared to a plurality of moments before the historical sample moment; The planning factor at the historical sample moment is determined according to the increment.

4. The method according to claim 3, characterized in that Constructing objective functions related to resource allocation and input costs includes: Determine the product of the resource allocation amount and the resource unit cost for each type of resource; The products of various resources are summed to obtain the objective function, which is as follows: Total Cost represents the total cost of resource allocation, T represents the total time required to predict and allocate resources in the future, and C i represents the resource unit cost of the i-th resource, x i (t) represents the amount of resources allocated at each time t, and N represents the number of types of configured resources.

5. The method according to claim 4, characterized in that Using the constraints, the objective function is optimized to obtain the load balancing optimization decision including: The objective function is constrained by the following formula to determine the load balancing optimization decision: The constraints are: x i (t) represents the amount of resources allocated at each time t, N represents the number of types of configured resources; R i (t) represents the predicted resource demand.

6. The method according to claim 5, characterized in that A penalty term is introduced to optimize the objective function and determine the load balancing optimization decision.

7. The method according to claim 5, characterized in that The method further includes: adjusting and configuring resources of various services according to the load balancing optimization decision, including: Determine the number of service nodes deployed in the cloud platform based on the proportion of various resources corresponding to each business in the load balancing optimization decision.

8. A cloud resource operation analysis device, characterized in that: The device comprises: A collection unit, used to collect load data of various services at multiple historical moments according to a preset period, and determine the planning factor of the historical moment according to the load data of various services at the multiple historical moments; A prediction result determination unit, used to input the load data of various services at historical moments and the planning factors into a resource usage prediction model, and obtain a resource usage prediction result at a future moment output by the model; wherein the resource usage prediction model is obtained by training an LSTM initial model based on the load sample data and sample planning factors of various services at multiple historical sample moments; An objective function construction unit, used to determine constraint conditions according to the resource usage prediction result, and to construct an objective function related to resource allocation and input cost; An optimization decision determination unit, used to optimize the objective function using the constraint conditions to obtain a load balancing optimization decision; The resource configuration unit is used to adjust and configure resources of various services according to the load balancing optimization decision.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images