Server resource dynamic allocation method and system

Abstract

The invention discloses a server resource dynamic allocation method and system, and relates to the technical field of electric digital data processing. The server resource dynamic allocation method comprises the following steps: S1, acquiring to-be-allocated server data; s2, the comprehensive evaluation model obtains a comprehensive load coefficient; and S3, the comprehensive value model obtains a comprehensive value. According to the invention, the comprehensive load coefficient is obtained through the comprehensive evaluation model, whether the comprehensive load coefficient exceeds the preset threshold is monitored, and if yes, resource allocation is carried out according to the comprehensive value of the to-be-allocated server obtained by the comprehensive value model, so that the effect of improving the resource utilization rate of the server is achieved; the problem that in the prior art, the server resource utilization rate is low is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a method and system for dynamically allocating server resources. Background Art

[0002] With the continuous development of cloud computing, the management and allocation of server resources have become key issues in improving system performance. Server resources include CPU, memory, storage and network bandwidth, etc. The effective use of these resources is directly related to the stability of cloud services and user experience. Dynamic resource allocation technology can adjust resource allocation in different time periods and load conditions by monitoring system load and real-time demand to ensure that each application and service can obtain sufficient computing power and storage space. The flexibility and efficiency of dynamic resource allocation enable the cloud computing environment to maximize the use of physical server resources, thereby improving the performance and reliability of the overall system and maximizing the efficiency of platform services.

[0003] The existing system detects and evaluates the cloud computing platform server by collecting relevant data of the cloud computing platform server, so as to reasonably and efficiently dynamically allocate resources and improve the resource utilization efficiency of the server. For example, the patent application with publication number: CN114968492A discloses a method and system for dynamically adjusting container resources in a cloud environment, including: determining the total resource load on the first server according to the resource usage; judging whether there are idle resources or resource load overload on the first server; migrating the container with the highest resource usage to the second server; using the first resource allocation strategy to allocate resources for the container without setting the resource upper limit; adjusting the resource allocation upper limit of the container, and adjusting the resources of the container using the third resource allocation strategy; according to the size of the idle resources not allocated to the container in the first server and the sum of all resources that need to be adjusted upward, allocating resources to the container from the idle resources not allocated to the container in the first server and the effective idle resources on the first server. For example, the invention patent with announcement number: CN115495231 B announces a dynamic resource scheduling method and system in complex scenarios with high concurrency tasks, including: the central server obtains the process information and resource occupancy status running on all resource nodes of the system, calculates the system resource utilization rate, and establishes the relationship between node load and resource utilization rate; obtains the system concurrent task process set, establishes a task level sequence, and allocates resources according to the node resource load and task level sequence.

[0004] In the above-mentioned prior art, there are at least the following technical problems: In the prior art, the cloud computing platform involves dynamic allocation of servers. During the allocation process, high load will lead to performance bottlenecks, resource competition and possible service interruption, and automatic expansion and load balancing are required to ensure the stability and response speed of the system. Low load will lead to idle resources and uneconomical costs, because different types of servers experience different levels of load in different time periods, and there is a problem of low server resource utilization. Summary of the invention

[0005] The present application aims to partially or completely solve the problem of low server resource utilization in the prior art by providing a method and system for dynamically allocating server resources, thereby improving the efficiency of server resource utilization.

[0006] In the first aspect, the present application provides a method for dynamic allocation of server resources, comprising the following steps: S1, collecting performance data and usage data of the server to be allocated during a preset monitoring period and storing them in a preset database, wherein the performance data include response time, request rate, throughput and error rate, and the usage data include CPU utilization, memory utilization and network bandwidth; S2, comprehensively evaluating the performance data of the server to be allocated through a comprehensive evaluation model to obtain a comprehensive load coefficient, and the comprehensive load coefficient is used to reflect the comprehensive performance of the server to be allocated; S3, monitoring the comprehensive load coefficient through a preset monitoring window, and if the comprehensive load coefficient exceeds a preset threshold, allocating resources to the server to be allocated according to the comprehensive value obtained by analyzing the usage data of the server to be allocated according to the comprehensive value model, and the comprehensive value is used to obtain the allocated resource amount of the server to be allocated.

[0007] Furthermore, the steps for obtaining the performance data are as follows: record the response time of the server to be assigned within a preset monitoring period, the response time indicating the time from the client sending a request to the server returning a response; record the number of requests processed by the server to be assigned within the preset monitoring period, and obtain the request rate by ratioing the number of requests with the detection period; record the amount of data transmitted by the server to be assigned within the preset monitoring period, and obtain the throughput by ratioing the amount of data transmitted with the detection period; record the number of error requests of the server to be assigned within the preset monitoring period, and obtain the error rate by ratioing the number of error requests with the number of requests.

[0008] Furthermore, the steps for obtaining the usage data are as follows: counting the CPU working time of the server to be allocated within the preset monitoring period, and ratioing the amount of CPU resources used by the server to be allocated with the duration corresponding to the preset monitoring period to obtain the CPU utilization rate of the server to be allocated; counting the memory usage of the server to be allocated within the preset monitoring period, and ratioing the memory usage of the server to be allocated with the total memory to obtain the memory utilization rate of the server to be allocated; recording the network transmission data volume of the server to be allocated within the preset monitoring period, and ratioing the network transmission data volume of the server to be allocated with the duration corresponding to the preset monitoring period to obtain the network bandwidth of the server to be allocated.

[0009] Furthermore, the performance data and usage data of the server to be assigned are collected during a preset monitoring period, and then data preprocessing is included. The process of the data preprocessing is as follows: deduplication is performed on the collected performance data and usage data according to a deduplication function; a box plot of the performance data and usage data is drawn according to a boxplot function, and outliers in the performance data and usage data are obtained according to the boxplot and removed.

[0010] Furthermore, the storage in the preset database also includes data transmission, and the data transmission process is as follows: the performance data and usage data after data preprocessing are divided into data blocks according to preset sizes and sent to the receiving end through the concurrent threads of the sending end, and at the same time, a confirmation response signal is listened to within a preset time period, and the confirmation response signal represents a confirmation signal sent by the receiving end after receiving the data block; if the sending end does not listen to the confirmation response signal within the preset time, the corresponding data block is resent and the listening continues.

[0011] Furthermore, the specific expression of the comprehensive load factor is:

[0012]

[0013] Where T represents the comprehensive load factor of the server, c j represents the average response time of the server to be assigned in the jth preset monitoring week, Δc j represents the reference response time of the jth preset monitoring cycle, θ1 represents the response time weight, and d j represents the average request rate of the server to be assigned in the jth preset monitoring week, Δd j represents the reference request rate of the jth preset monitoring period, θ2 represents the request rate weight, and a j represents the average throughput of the server to be allocated in the jth preset monitoring week, Δa j represents the reference throughput of the jth preset monitoring period, θ3 represents the throughput weight, and f jrepresents the average error rate of the server to be assigned in the jth preset monitoring week, Δf j represents the reference error rate of the jth preset monitoring period, θ4 represents the error rate weight factor, j represents the number of the preset monitoring period, j=1,2,3,…m, and m represents the total number of preset monitoring periods.

[0014] Furthermore, the process of obtaining the comprehensive value is as follows: the CPU utilization, memory utilization and network bandwidth of the server to be allocated and the standard CPU utilization, standard memory utilization and standard network bandwidth are input into the comprehensive value model to obtain the comprehensive value of the server to be allocated, the standard CPU utilization is based on the average value of historical CPU utilization data obtained from the preset database, the standard memory utilization is based on the average value of historical memory utilization data obtained from the preset database, and the standard network bandwidth is based on the average value of historical network bandwidth data obtained from the preset database. The specific expression of the comprehensive value is:

[0015]

[0016] Where k represents the number of the server to be allocated, k = 1, 2, 3, ... Q, Q is the total number of servers to be allocated, S k is the comprehensive value of the kth server to be assigned, n is the number of unit time, n=

[0017] 1,2,3,…N, N is the total number of unit time in the preset monitoring period, A n→k represents the CPU utilization of the kth server to be allocated in the nth unit time, A is the standard CPU utilization, μ1 represents the set CPU utilization weight, and B n→k represents the memory usage of the kth server to be allocated in the nth unit time, B is the standard memory usage, μ2 represents the set memory usage weight, C n→k represents the network bandwidth of the kth server to be allocated in the nth unit time, C is the standard network bandwidth, μ3 represents the weight of the set network bandwidth, σ is the maximum deviation value between the CPU utilization of the server to be allocated and the standard CPU utilization, and e represents a natural constant.

[0018] Furthermore, the specific steps for allocating resources to the servers to be allocated based on the comprehensive value obtained by analyzing the usage data of the servers to be allocated according to the comprehensive value model are as follows: adding the comprehensive values ​​of all the servers to be allocated to obtain the comprehensive value of the total server, and performing ratio calculations on the comprehensive values ​​of the servers to be allocated and the comprehensive value of the total server to obtain the corresponding allocation ratio, and respectively configuring the resources of the servers to be allocated according to the allocation ratio.

[0019] Furthermore, the specific steps of monitoring the comprehensive load coefficient in the preset monitoring window are as follows: comparing the numerical relationship between the comprehensive load coefficient and the preset threshold through the preset monitoring window: if the comprehensive load coefficient exceeds the preset threshold, recalculate the resource allocation amount after obtaining the comprehensive value of the server to be allocated according to the comprehensive value model, and continue to monitor the comprehensive load coefficient in the preset monitoring window; if the comprehensive load coefficient does not exceed the preset threshold, continue to monitor the comprehensive load coefficient in the preset monitoring window.

[0020] In a second aspect, the present application provides a server resource dynamic allocation system, which implements any server resource dynamic allocation method described in the first aspect, including: the data acquisition module, the comprehensive evaluation module and the resource allocation module:

[0021] The data collection module is used to collect performance data and usage data of the server to be allocated in a preset monitoring period and store them in a preset database, the performance data includes response time, request rate, throughput and error rate, and the usage data includes CPU utilization, memory utilization and network bandwidth;

[0022] The comprehensive evaluation module is used to comprehensively evaluate the performance data of the server to be assigned through a comprehensive evaluation model to obtain a comprehensive load factor, and the comprehensive load factor is used to reflect the comprehensive performance of the server to be assigned;

[0023] The resource allocation module is used to monitor the comprehensive load coefficient through a preset monitoring window. If the comprehensive load coefficient exceeds a preset threshold, resources are allocated to the server to be allocated based on a comprehensive value obtained by analyzing the usage data of the server to be allocated according to a comprehensive value model. The comprehensive value is used to obtain the allocated resource amount of the server to be allocated.

[0024] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0025] 1. The load of the server to be allocated is evaluated by inputting the usage data of the server to be allocated into the comprehensive evaluation model, and then the performance data of the server to be allocated is input into the comprehensive value model to obtain the comprehensive value of the server to be allocated. Resources are allocated according to the comprehensive value of the server to be allocated, thereby improving the accuracy of resource allocation, thereby achieving an improvement in the efficiency of server resource utilization and effectively solving the problem of low server resource utilization.

[0026] 2. Through data preprocessing, the performance data and usage data of the allocated servers are deduplicated and outliers are removed, thereby improving the accuracy of the performance data and usage data, and thus achieving more accurate load prediction and resource allocation.

[0027] 3. The performance data and usage data of the assigned server are transmitted through blocks and concurrent threads, thereby reducing data loss and transmission errors during the data transmission process, thereby achieving transmission reliability and integrity. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flow chart of a method for dynamically allocating server resources provided in this application;

[0029] Figure 2 A structural diagram of a server resource dynamic allocation system provided for this application;

[0030] Figure 3 This is a graph showing the changing trend of the comprehensive value provided for this application. DETAILED DESCRIPTION

[0031] The present application solves the problem of low server resource utilization in the prior art by providing a method and system for dynamic allocation of server resources. The method collects and stores performance data and usage data of the server to be allocated within a preset monitoring period, and then uses a comprehensive evaluation model to evaluate the usage data of the server to be allocated to obtain a comprehensive load coefficient to reflect the overall performance of the server. Finally, the comprehensive load coefficient is monitored. When it exceeds a preset threshold, the performance data of the server to be allocated is analyzed according to the comprehensive value model to determine a comprehensive value, and resources are allocated to the server to be allocated according to the comprehensive value, thereby achieving the effect of improving server resource utilization.

[0032] The technical solution in this application is to solve the problem of low server resource utilization. The overall idea is as follows: obtain the comprehensive load factor through a comprehensive evaluation model, monitor whether the comprehensive load factor exceeds a preset threshold, and if it exceeds the preset threshold, allocate resources according to the comprehensive value of the server to be allocated obtained by the comprehensive value model, thereby improving the server resource utilization. In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the drawings and specific implementation methods of the specification.

[0033] First, as Figure 1 FIG. 1 is a flow chart of a method for dynamically allocating server resources provided by the present application, the method comprising the following steps:

[0034] S1, collects performance data and usage data of the server to be allocated in a preset monitoring period and stores them in a preset database, the performance data includes response time, request rate, throughput and error rate, and the usage data includes CPU utilization, memory utilization and network bandwidth;

[0035] S2, comprehensively evaluating the performance data of the server to be assigned through a comprehensive evaluation model to obtain a comprehensive load coefficient, which is used to reflect the comprehensive performance of the server to be assigned;

[0036] S3, monitor the comprehensive load coefficient through a preset monitoring window. If the comprehensive load coefficient exceeds a preset threshold, resources are allocated to the server to be allocated based on a comprehensive value obtained by analyzing the usage data of the server to be allocated according to a comprehensive value model. The comprehensive value is used to obtain the amount of allocated resources for the server to be allocated.

[0037] In this embodiment, the server to be allocated is a server in the cloud computing platform, such as a database server or an application server; the performance data is used to describe the operation status of the server to be allocated, the comprehensive evaluation model is used to obtain the comprehensive load coefficient, the comprehensive value model is used to obtain the comprehensive value, and the preset threshold is obtained by inputting the mean of the historical performance data into the comprehensive evaluation model; the server resources are dynamically adjusted according to the comprehensive load coefficient and the comprehensive value, thereby reducing the risk of server overload and achieving optimal allocation of resources and system stability.

[0038] Furthermore, the specific expression of the comprehensive load factor is:

[0039]

[0040] Where T represents the comprehensive load factor, c j represents the average response time of the server to be assigned in the jth preset monitoring week, Δc j represents the reference response time of the jth preset monitoring cycle, θ1 represents the response time weight, and d j represents the average request rate of the server to be assigned in the jth preset monitoring week, Δd j represents the reference request rate of the jth preset monitoring period, θ2 represents the request rate weight, and a j represents the average throughput of the server to be allocated in the jth preset monitoring week, Δa j represents the reference throughput of the jth preset monitoring period, θ3 represents the throughput weight, and f j represents the average error rate of the server to be assigned in the jth preset monitoring week, Δf j represents the reference error rate of the jth preset monitoring period, θ4 represents the error rate weight, j represents the number of the preset monitoring period, j=1,2,3,…m, and m represents the total number of preset monitoring periods.

[0041] It is important to understand that T is the comprehensive load factor. is the difference between the average response time of the server to be assigned and the reference response time in the jth preset monitoring period, is the difference between the average request rate of the server to be assigned in the jth preset monitoring week and the reference request rate, is the difference between the average throughput of the server to be assigned in the jth preset monitoring week and the reference throughput, It is the difference between the average error rate of the server to be assigned in the jth preset monitoring week and the reference error rate. The comprehensive load coefficient is affected by the difference between the response time, request rate, throughput and error rate and their respective reference values. The faster the response time, the lower the error rate, the higher the request rate and the greater the throughput, the smaller the corresponding comprehensive load coefficient T, indicating that the comprehensive performance of the server to be assigned is good; it realizes more accurate acquisition of the comprehensive load coefficient, thereby more accurately and intuitively reflecting the load situation of the server to be assigned.

[0042] It should be understood that the response time weight is the impact factor corresponding to the response time, which indicates the numerical value of the impact of the response time on the comprehensive value. When in use, the impact factor corresponding to the response time can be directly obtained from the preset database, and the corresponding relationship can be a preset mapping relationship. For example, the response time data and the weight corresponding to the response time value in the preset database form a mapping set. After the real-time response time is input into the mapping set, the corresponding weight will be obtained. The mapping relationship can be a one-to-one correspondence, a one-to-many relationship, or a many-to-one relationship. In this example, its value range is [0,1]; the request rate weight is the impact factor corresponding to the request rate, which indicates the numerical value of the impact of the request rate on the comprehensive value. When in use, the impact factor corresponding to the request rate can be directly obtained from the preset database, and the corresponding relationship can be a preset mapping relationship. For example, the request rate data and the impact factor in the database form a mapping set, and the real-time request rate is converted into the corresponding impact factor through this mapping set. The mapping relationship may be in the form of one-to-one correspondence, one-to-many or many-to-one. In this example, the value range of the impact factor is [0,1]. The throughput weight is the impact factor corresponding to the throughput, which indicates the numerical value of the impact degree of the throughput on the comprehensive value. When used, the impact factor corresponding to the throughput can be directly obtained from the preset database. The corresponding relationship may be a preset mapping relationship. For example, the throughput data and the impact factor in the database form a mapping set, and the real-time throughput is converted into the corresponding impact factor through this mapping set. The mapping relationship may be in the form of one-to-one correspondence, one-to-many or many-to-one. In this example, the value range of the impact factor is [0,1]. The error rate weight is the impact factor corresponding to the error rate, which indicates the numerical value of the impact degree of the error rate on the comprehensive value. When used, the impact factor corresponding to the error rate can be directly obtained from the preset database. The corresponding relationship may be a preset mapping relationship. The error rate data and the impact factor in the database form a mapping set. The real-time error rate is converted into the corresponding impact factor through this mapping set. The mapping relationship may be in the form of one-to-one correspondence, one-to-many or many-to-one. In this example, the value range of the impact factor is [0,1].

[0043] In this embodiment, the response time score is defined as Used to evaluate the response time and error rate score of the server to be assigned Used to evaluate the error rate and request throughput score of the server to be assigned Used to evaluate the request rate and throughput of the server to be assigned. G is a multi-index evaluation value, G = G j1 +G j2 +G j3 , which is used to evaluate the comprehensive performance of the server to be allocated within a specific monitoring period. Assuming that the total number of preset monitoring periods is 1, the response time weight is 0.5, the request rate weight is 0.3, the throughput weight is 0.2, and the error rate weight is 0.4, the change statistics of the comprehensive load factor corresponding to the server to be allocated are shown in Table 1:

[0044] Table 1 Statistics of changes in comprehensive load factor

[0045]

[0046] As can be seen from Table 1, the first group has the lowest multi-index evaluation value and the lowest comprehensive load factor, while the third group has the highest multi-index evaluation value and the highest comprehensive load factor; Table 1 clearly shows that G increases with G j1 , G j2 and G j3 There is a positive correlation between the multiple index evaluation values ​​and G j1 , G j2 and G j3 The multi-index evaluation value G increases with the increase of the comprehensive load coefficient; there is a positive correlation between the multi-index evaluation value and the comprehensive load coefficient. At the same time, the comprehensive load coefficient is negatively correlated with the comprehensive performance of the server to be allocated. The lower the multi-index evaluation value G is, the lower the comprehensive load coefficient is, which means that the comprehensive performance of the server to be allocated is better; by defining the multi-index evaluation value G and the comprehensive load coefficient and showing the correlation between them, it is judged whether to allocate resources to the server to be allocated, thereby achieving more accurate and more effective resource allocation.

[0047] Furthermore, the steps for obtaining performance data are as follows: record the response time of the server to be assigned within the preset monitoring period, the response time represents the time from the client sending the request to the server returning the response; record the number of requests processed by the server to be assigned within the preset monitoring period, and obtain the request rate by ratioing the number of requests with the detection period; record the amount of data transmitted by the server to be assigned within the preset monitoring period, and obtain the throughput by ratioing the amount of data transmitted with the detection period; record the number of error requests of the server to be assigned within the preset monitoring period, and obtain the error rate by ratioing the number of error requests with the number of requests.

[0048] In this embodiment, data is collected based on Splunk's log report of the server to be assigned, including the response time of client requests to the server to be assigned, the number of requests processed by the server to be assigned, and the total amount of data transmitted by the server to be assigned within a preset monitoring period. The number of requests processed by the server to be assigned and the total amount of data transmitted by the server to be assigned are divided by the length of the preset monitoring period to obtain the corresponding request rate and throughput. The response speed of the server is obtained through the response time, the load capacity of the server is evaluated through the request rate, the data processing capacity of the server is evaluated through the throughput, and the stability of the server to be assigned is evaluated through the error rate, thereby realizing dynamic adjustment of the resources of the server to be assigned and improving the operating efficiency and stability of the server to be assigned.

[0049] Furthermore, the steps for obtaining usage data are as follows: counting the CPU working time of the server to be assigned within the preset monitoring period, and ratioing the amount of CPU resources used by the server to be assigned with the duration corresponding to the preset monitoring period to obtain the CPU utilization rate of the server to be assigned; counting the memory usage of the server to be assigned within the preset monitoring period, and ratioing the memory usage of the server to be assigned with the total memory to obtain the memory utilization rate of the server to be assigned; recording the network transmission data volume of the server to be assigned within the preset monitoring period, and ratioing the network transmission data volume of the server to be assigned with the duration corresponding to the preset monitoring period to obtain the network bandwidth of the server to be assigned.

[0050] In this embodiment, the sar command is used to record the CPU usage data, memory usage data and network transmission data volume of the server to be allocated within a preset monitoring period, and a corresponding log file is automatically generated. The data in the log file is exported to extract the CPU usage data and the duration of the monitoring period to obtain the CPU utilization rate, extract the memory usage data and the duration of the monitoring period to obtain the memory usage rate, and extract the network transmission data volume and the duration of the monitoring period to obtain the network bandwidth; the CPU utilization rate, memory usage rate and network bandwidth are obtained by the sar command, thereby achieving more efficient and accurate acquisition of usage data.

[0051] Furthermore, the performance data and usage data of the server to be assigned are collected during a preset monitoring period, followed by data preprocessing, the process of which is as follows: deduplication is performed on the collected performance data and usage data according to a deduplication function; a box plot of the performance data and usage data is drawn according to a boxplot function, and outliers in the performance data and usage data are obtained and removed according to the boxplot.

[0052] In this embodiment, the drop_duplicates() function in the Pandas library is imported into Python to remove completely duplicate data in the performance data and usage data; the median, quartiles and interquartile range of the performance data and usage data are calculated by using Python's matplotlib and seaborn libraries, and a boxplot of the performance data and usage data is drawn by the boxplot function, in which the quartiles are the boxes of the boxplot, and the median is the center line of the box of the boxplot. If the data in the performance data and usage data exceeds 1.5 times the quartile value, it is recorded as an outlier and deleted; the distribution of the performance data and usage data is intuitively displayed through the boxplot, so as to accurately remove the outliers in the performance data and usage data, thereby improving the reliability of the performance data and usage data.

[0053] Furthermore, it is stored in a preset database, which also includes data transmission. The data transmission process is as follows: the performance data and usage data after data preprocessing are divided into data blocks according to a preset size and sent to the receiving end through the concurrent thread of the sending end, and at the same time, a confirmation response signal is listened to within a preset time period. The confirmation response signal represents a confirmation signal sent by the receiving end after receiving the data block; if the sending end does not listen to the confirmation response signal within the preset time, it resends the corresponding data block and continues to listen.

[0054] In this embodiment, usage data and performance data are divided into blocks according to network bandwidth, network protocol is used for data transmission, threads are created through thread i ng module, and thread i ng.Lock() is used to synchronize threads to prevent conflicts between threads in concurrent thread transmission; multiple data blocks are transmitted simultaneously through multi-threaded concurrent processing, thereby speeding up data transmission, maximizing the use of network bandwidth and processor resources, and improving the performance and response speed of the overall system. Thread i ng.Lock() is used to synchronize threads to improve the reliability of data transmission.

[0055] Furthermore, the process of obtaining the comprehensive value is as follows: the CPU utilization, memory utilization and network bandwidth of the server to be allocated and the standard CPU utilization, standard memory utilization and standard network bandwidth are input into the comprehensive value model to obtain the comprehensive value of the server to be allocated. The standard CPU utilization is obtained according to the average value of the historical CPU utilization data in the preset database, the standard memory utilization is obtained according to the average value of the historical memory utilization data in the preset database, and the standard network bandwidth is obtained according to the average value of the historical network bandwidth data in the preset database. The specific expression of the comprehensive value is:

[0056]

[0057] Where k represents the number of the server to be allocated, k = 1, 2, 3, ... Q, Q is the total number of servers to be allocated, S k is the comprehensive value of the kth server to be assigned, n is the number of unit time, n=

[0058] 1,2,3,…N, N is the total number of unit time in the preset monitoring period, A n→k represents the CPU utilization of the kth server to be allocated in the nth unit time, A is the standard CPU utilization, μ1 represents the set CPU utilization weight, and B n→k represents the memory usage of the kth server to be allocated in the nth unit time, B is the standard memory usage, μ2 represents the set memory usage weight, C n→k represents the network bandwidth of the kth server to be allocated in the nth unit time, C is the standard network bandwidth, μ3 represents the weight of the set network bandwidth, σ is the maximum deviation value between the CPU utilization of the server to be allocated and the standard CPU utilization, and e represents a natural constant.

[0059] It should be understood that the CPU utilization weight is the impact factor corresponding to the CPU utilization, which indicates the numerical value of the impact of the CPU utilization on the comprehensive value. When in use, the impact factor corresponding to the CPU utilization can be directly obtained from the preset database, and the corresponding relationship can be a preset mapping relationship. For example, the CPU utilization data and the impact factors in the database form a mapping set, and the real-time CPU utilization is converted into the corresponding impact factor through this mapping set. The mapping relationship can be in the form of one-to-one correspondence, one-to-many, or many-to-one. In this example, the value range of the impact factor is [0,1]; the memory usage weight is the impact factor corresponding to the memory usage, which indicates the numerical value of the impact of the memory usage on the comprehensive value. When in use, the impact factor corresponding to the memory usage can be directly obtained from the preset database, and the corresponding relationship can be a preset mapping relationship. For example, the memory usage data and the influencing factors in the database form a mapping set, and the real-time memory usage is converted into the corresponding influencing factors through this mapping set. The mapping relationship can be in the form of one-to-one correspondence, one-to-many or many-to-one. In this example, the value range of the influencing factor is [0,1]; the network bandwidth weight is the influencing factor corresponding to the network bandwidth, which represents the numerical value of the influence of the network bandwidth on the comprehensive value. When used, the influencing factor corresponding to the network bandwidth can be directly obtained from the preset database, and the corresponding relationship can be a preset mapping relationship. For example, the network bandwidth data and the influencing factors in the database form a mapping set, and the real-time network bandwidth is converted into the corresponding influencing factor through this mapping set. The mapping relationship can be in the form of one-to-one correspondence, one-to-many or many-to-one. In this example, the value range of the influencing factor is [0,1].

[0060] It is important to understand that Figure 2 As shown, it is the trend chart of the comprehensive value of the server to be allocated, corresponding to |A n→k -A|>σ, define It represents the deviation of the memory usage of the kth server to be allocated in the nth unit time. It indicates the deviation of the network bandwidth of the kth server to be allocated in the nth unit time. The comprehensive value is positively correlated with the deviation of the network bandwidth. The comprehensive value increases with the increase of the network bandwidth deviation. When the system recognizes that the network bandwidth is insufficient or unstable, the resource request of the server to be allocated will be regarded as a higher priority; the comprehensive value is positively correlated with the deviation of the memory usage rate. The comprehensive value increases with the increase of the deviation of the memory usage rate. When the system recognizes that the memory resources are insufficient, the priority of the server to be allocated or the resource request will be increased. The comprehensive value is proportional to the amount of resources required by the server to be allocated. The greater the deviation of the memory usage rate, the greater the deviation of the network bandwidth, and the greater the comprehensive value, the more resources the server to be allocated requires. In the case of limited resources, priority is given to those with high demand, which improves the efficiency of resource allocation.

[0061] In this embodiment, the comprehensive value of the server to be allocated is obtained in two cases. The first case is when |A n→k -A|>σ, the comprehensive value is obtained based on two parts. The CPU utilization deviation value has exceeded the maximum value and cannot be used as a factor in evaluating the comprehensive value. The first part is The second part is S k It is obtained based on the deviation of memory usage and network bandwidth from their respective standard resource utilization; the second case is in |A n→k -A|≤σ, the comprehensive value is obtained based on three parts. The first part is The second part is The third part is S k It is obtained based on the deviation of CPU utilization, memory utilization and network bandwidth from their respective standard resource utilization; the comprehensive value calculation model achieves improved accuracy and efficiency of resource allocation by comprehensively considering the usage of CPU utilization, memory utilization and network bandwidth and their deviations from standard resource usage.

[0062] Furthermore, the specific steps for allocating resources to the servers to be allocated based on the comprehensive value obtained by analyzing the usage data of the servers to be allocated according to the comprehensive value model are as follows: adding the comprehensive values ​​of all the servers to be allocated to obtain the comprehensive value of the total server, and performing ratio calculations on the comprehensive values ​​of the servers to be allocated and the comprehensive value of the total server to obtain the corresponding allocation ratio, and respectively configuring the resources of the servers to be allocated according to the allocation ratio.

[0063] In this embodiment, the ratio obtained by the ratio of the comprehensive value of the server to be allocated to the comprehensive value of the total servers reflects the weight of the corresponding server to be allocated among all the servers to be allocated. There are three servers to be allocated, and 1000 units of CPU are available for allocation. The comprehensive value of the first server to be allocated is 10, the comprehensive value of the second server to be allocated is 20, and the comprehensive value of the third server to be allocated is 20. The total comprehensive value of the total servers is 50. The comprehensive value of the first server to be allocated is ratioed to the comprehensive value of the total servers, and the proportion of the first server to be allocated is 0.2, so the first server to be allocated is allocated 200 units of CPU; the total amount of resources is allocated to the servers to be allocated according to the proportion of the servers to be allocated; through proportional allocation based on the comprehensive value, dynamic allocation of resources is achieved and the overall resource utilization is optimized.

[0064] Furthermore, the specific steps of monitoring the comprehensive load coefficient in the preset monitoring window are as follows: compare the numerical relationship between the comprehensive load coefficient and the preset threshold through the preset monitoring window: if the comprehensive load coefficient exceeds the preset threshold, recalculate the resource allocation amount after obtaining the comprehensive value of the server to be allocated according to the comprehensive value model, and continue to monitor the comprehensive load coefficient in the preset monitoring window; if the comprehensive load coefficient does not exceed the preset threshold, continue to monitor the comprehensive load coefficient in the preset monitoring window.

[0065] In this embodiment, the preset threshold is calculated by averaging the performance data in the preset database. The average response time is the sum of all response times divided by the number of responses. The average request rate is the sum of the number of requests in all time periods divided by the total number of time periods. The average throughput is the sum of the throughput in all time periods divided by the total number of time periods. The average error rate is the sum of the number of error requests divided by the total number of requests. The calculation results are brought into the comprehensive evaluation model to obtain the preset threshold. The refined management of the load is achieved through real-time monitoring and dynamic adjustment of resource allocation, thereby improving resource utilization efficiency.

[0066] Second, as Figure 3 As shown, it is a structural diagram of a server resource dynamic allocation system provided by the present application. The server resource dynamic allocation system provided by the present application implements any server resource dynamic allocation method described in the first aspect, including: a data acquisition module, a comprehensive evaluation module and a resource allocation module:

[0067] The data collection module is used to collect performance data and usage data of the server to be allocated in a preset monitoring period and store them in a preset database. The performance data includes response time, request rate, throughput and error rate, and the usage data includes CPU utilization, memory utilization and network bandwidth.

[0068] The comprehensive evaluation module is used to comprehensively evaluate the performance data of the server to be assigned through a comprehensive evaluation model to obtain a comprehensive load factor, and the comprehensive load factor is used to reflect the comprehensive performance of the server to be assigned;

[0069] The resource allocation module is used to monitor the comprehensive load coefficient through a preset monitoring window. If the comprehensive load coefficient exceeds the preset threshold, resources are allocated to the server to be allocated based on the comprehensive value obtained by analyzing the usage data of the server to be allocated according to the comprehensive value model. The comprehensive value is used to obtain the allocated resource amount of the server to be allocated.

[0070] In this embodiment, if the comprehensive load coefficient does not exceed the preset threshold, the comprehensive load coefficient continues to be monitored. By monitoring the comprehensive load coefficient, real-time tracking of the performance of the assigned server is ensured, ensuring the stability of the system and the efficiency of resource allocation, and realizing continuous performance monitoring and dynamic resource adjustment mechanism to ensure effective resource allocation and management according to the real-time load conditions.

[0071] To summarize, the present application evaluates the load conditions of the servers to be allocated by inputting the usage data of the servers to be allocated into a comprehensive evaluation model, and then inputs the performance data of the servers to be allocated into a comprehensive value model to obtain the comprehensive value of the servers to be allocated, and allocates resources according to the comprehensive value of the servers to be allocated, thereby improving the accuracy of resource allocation, and further achieving an improvement in the efficiency of server resource utilization, effectively solving the problem of low server resource utilization.

[0072] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0076] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0077] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for dynamically allocating server resources, characterized in that: The following steps are involved: S1, collecting performance data and usage data of the server to be allocated in a preset monitoring period and storing them in a preset database, wherein the performance data includes response time, request rate, throughput and error rate, and the usage data includes CPU utilization, memory utilization and network bandwidth; S2, comprehensively evaluating the usage data of the server to be assigned through a comprehensive evaluation model to obtain a comprehensive load coefficient, where the comprehensive load coefficient is used to reflect the comprehensive performance of the server to be assigned; S3, monitor the comprehensive load coefficient through a preset monitoring window. If the comprehensive load coefficient exceeds a preset threshold, resources are allocated to the server to be allocated based on a comprehensive value obtained by analyzing the performance data of the server to be allocated according to a comprehensive value model. The comprehensive value is used to obtain the amount of allocated resources for the server to be allocated.

2. A method for dynamically allocating server resources as claimed in claim 1, characterized in that: The steps for obtaining the performance data are as follows: Record the time from when the client sends a request to when the server returns a response within a preset monitoring period to obtain the response time; Record the number of requests to be processed by the assigned server within the preset monitoring period, and calculate the ratio of the number of requests to the detection period to obtain the request rate; Record the amount of data transmitted by the server to be assigned within the preset monitoring period, and calculate the ratio of the amount of data transmitted to the detection period to obtain the throughput; The number of error requests to the server to be assigned within the preset monitoring period is recorded, and the error rate is obtained by calculating the ratio of the number of error requests to the number of requests.

3. A method for dynamically allocating server resources as claimed in claim 1, characterized in that: The steps for obtaining the usage data are as follows: Count the CPU working time of the server to be allocated within the preset monitoring period, and calculate the ratio of the CPU resources used by the server to be allocated to the duration corresponding to the preset monitoring period to obtain the CPU utilization rate of the server to be allocated; Count the memory usage of the server to be allocated within a preset monitoring period, and calculate the ratio of the memory usage of the server to be allocated to the total memory to obtain the memory usage rate of the server to be allocated; The network transmission data volume of the server to be allocated within the preset monitoring period is recorded, and the network bandwidth of the server to be allocated is obtained by performing a ratio operation on the network transmission data volume of the server to be allocated and the duration corresponding to the preset monitoring period.

4. A method for dynamically allocating server resources as claimed in claim 1, characterized in that: The performance data and usage data of the server to be allocated are collected in a preset monitoring period, and then data preprocessing is included. The process of data preprocessing is as follows: Deduplication is performed on the collected performance data and usage data according to the deduplication function; The boxplot function is used to draw boxplots of the performance data and usage data, and outliers in the performance data and usage data are obtained and removed based on the boxplots.

5. A method for dynamically allocating server resources as claimed in claim 4, characterized in that: The storage in the preset database also includes data transmission, and the process of data transmission is as follows: The performance data and usage data after data preprocessing are divided into blocks according to a preset size to obtain data blocks and sent to the receiving end through the concurrent thread of the sending end, and at the same time, a confirmation response signal is monitored within a preset time period, wherein the confirmation response signal represents a confirmation signal sent by the receiving end after receiving the data block; If the sending end does not hear the confirmation response signal within the preset time, it will resend the corresponding data block and continue to listen.

6. A method for dynamically allocating server resources as claimed in claim 1, characterized in that: The specific expression of the comprehensive load factor is: Where T represents the comprehensive load factor of the server, c j represents the average response time of the server to be assigned in the jth preset monitoring week, Δc j represents the reference response time of the jth preset monitoring cycle, θ1 represents the response time weight, and d j represents the average request rate of the server to be assigned in the jth preset monitoring week, Δd j represents the reference request rate of the jth preset monitoring period, θ2 represents the request rate weight, and a j represents the average throughput of the server to be allocated in the jth preset monitoring week, Δa j represents the reference throughput of the jth preset monitoring period, θ3 represents the throughput weight, and f j represents the average error rate of the server to be assigned in the jth preset monitoring week, Δf j represents the reference error rate of the jth preset monitoring period, θ4 represents the error rate weight factor, j represents the number of the preset monitoring period, j=1,2,3,…m, and m represents the total number of preset monitoring periods.

7. A method for dynamically allocating server resources as claimed in claim 1, characterized in that: The process of obtaining the comprehensive value is as follows: The CPU utilization, memory utilization and network bandwidth of the server to be allocated and the standard CPU utilization, standard memory utilization and standard network bandwidth are input into the comprehensive value model to obtain the comprehensive value of the server to be allocated. The standard CPU utilization is obtained according to the mean value of historical CPU utilization data in the preset database, the standard memory utilization is obtained according to the mean value of historical memory utilization data in the preset database, and the standard network bandwidth is obtained according to the mean value of historical network bandwidth data in the preset database. The specific expression of the comprehensive value is: Where k represents the number of the server to be allocated, k = 1, 2, 3, ... Q, Q is the total number of servers to be allocated, S k is the comprehensive value of the kth server to be assigned, n is the number of the unit time, n = 1, 2, 3, ... N, N is the total number of unit time in the preset monitoring period, A n→k represents the CPU utilization of the kth server to be allocated in the nth unit time, A is the standard CPU utilization, μ1 represents the set CPU utilization weight, and B n→k represents the memory usage of the kth server to be allocated in the nth unit time, B is the standard memory usage, μ2 represents the set memory usage weight, C n→k represents the network bandwidth of the kth server to be allocated in the nth unit time, C is the standard network bandwidth, μ3 represents the weight of the set network bandwidth, σ is the maximum deviation value between the CPU utilization of the server to be allocated and the standard CPU utilization, and e represents a natural constant.

8. A method for dynamically allocating server resources as claimed in claim 7, characterized in that: The specific steps of allocating resources to the allocated server according to the comprehensive value obtained by analyzing the performance data of the allocated server according to the comprehensive value model are as follows: The comprehensive values ​​of all servers to be allocated are added together to obtain the comprehensive value of the total server, and the comprehensive value of the server to be allocated and the comprehensive value of the total server are respectively ratioed to obtain the corresponding allocation ratio, and the resources of the server to be allocated are respectively configured according to the allocation ratio.

9. A method for dynamically allocating server resources as claimed in claim 8, characterized in that: The specific steps of monitoring the comprehensive load factor in the preset monitoring window are as follows: The relationship between the comprehensive load factor and the preset threshold is compared through a preset monitoring window. If the comprehensive load factor exceeds the preset threshold, the resource allocation amount is recalculated after obtaining the comprehensive value of the server to be allocated according to the comprehensive value model, and the comprehensive load factor is continuously monitored in the preset monitoring window. If the comprehensive load factor does not exceed the preset threshold, the comprehensive load factor continues to be monitored in the preset monitoring window.

10. A server resource dynamic allocation system, implementing the server resource dynamic allocation method according to any one of claims 1 to 9, characterized in that: include: Data collection module, comprehensive evaluation module and resource allocation module: The data collection module is used to collect performance data and usage data of the server to be allocated in a preset monitoring period and store them in a preset database, the performance data includes response time, request rate, throughput and error rate, and the usage data includes CPU utilization, memory utilization and network bandwidth; The comprehensive evaluation module is used to comprehensively evaluate the performance data of the server to be assigned through a comprehensive evaluation model to obtain a comprehensive load factor, and the comprehensive load factor is used to reflect the comprehensive performance of the server to be assigned; The resource allocation module is used to monitor the comprehensive load coefficient through a preset monitoring window. If the comprehensive load coefficient exceeds a preset threshold, resources are allocated to the server to be allocated based on a comprehensive value obtained by analyzing the usage data of the server to be allocated according to a comprehensive value model. The comprehensive value is used to obtain the allocated resource amount of the server to be allocated.

Images