Resource allocation method, apparatus, and electronic device

Abstract

The present disclosure provides a resource allocation method, device and electronic equipment. The resource allocation method comprises: determining the similarity between a plurality of nodes according to a first performance data time sequence of the nodes, forming a topology graph according to nodes with a similarity greater than a preset value, the nodes being to-be-assigned service nodes, and the first performance data time sequence being a sequence of a plurality of performance data of the to-be-assigned service nodes changing over time; determining the correlation coefficient between adjacent nodes in the topology graph, determining a second performance data time sequence of each node according to the correlation coefficient; determining an autocorrelation function of the node according to the second performance data time sequence; determining performance prediction data of the node according to the autocorrelation function and the second performance data time sequence; and determining a task node among the plurality of nodes according to the performance prediction data. The present disclosure can improve the utilization efficiency of cloud resources.

Description

Technical Field

[0001] The present disclosure relates to the field of cloud computing technology, and in particular to a resource allocation method, device, and electronic device. Background Art

[0002] With the rapid development of cloud computing technology, cloud resource scheduling has become a research hotspot in the field. Microservices architecture, as a mainstream application type in cloud-native environments, offers significant advantages in deployment and scalability due to its loosely coupled structure. However, this distributed and loosely coupled nature also presents new challenges for cloud resource scheduling. Traditional cloud resource scheduling methods often rely on statistical data or simple rules to make decisions, but often perform poorly when faced with the resource usage of large-scale, complex microservice applications. How to effectively allocate computing resources to these microservices, improve cloud resource utilization, and ensure their efficient and stable operation has become a pressing issue.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0004] The present disclosure aims to provide a resource allocation method, apparatus, and electronic device for efficiently allocating computing resources for at least microservices and other applications served by cloud computing, thereby improving cloud resource utilization.

[0005] According to a first aspect of an embodiment of the present disclosure, a resource allocation method is provided, comprising: determining similarities between a plurality of nodes based on a first performance data time series of the nodes, forming a topological graph based on nodes whose similarities are greater than a preset value, the nodes being service nodes to be allocated, and the first performance data time series being a sequence of multiple performance data of the service nodes to be allocated changing over time; determining a correlation coefficient between adjacent nodes in the topological graph, determining a second performance data time series of each of the nodes based on the correlation coefficient; determining an autocorrelation function of the node based on the second performance data time series; determining performance prediction data of the node based on the autocorrelation function and the second performance data time series; and determining a task node among a plurality of nodes based on the performance prediction data.

[0006] In an exemplary embodiment of the present disclosure, determining similarities between multiple nodes based on a time series of first performance data of the nodes includes:

[0007] Obtain a first performance data time series of the first node and a first performance data time series of the second node;

[0008] Calculate the dynamic time warping distance between the first performance data time series of the first node and the first performance data time series of the second node using a dynamic time warping algorithm;

[0009] The similarity between the first node and the second node is determined according to the dynamic warping distance, and the smaller the dynamic warping distance is, the greater the similarity is.

[0010] In an exemplary embodiment of the present disclosure, determining a correlation coefficient between adjacent nodes in the topological graph, and determining a second performance data time series of each node according to the correlation coefficient includes:

[0011] Determine an attention coefficient of a target node to each adjacent node using a preset loss function and the first performance data time series;

[0012] Normalizing each attention coefficient corresponding to the target node to obtain a correlation coefficient between the target node and each adjacent node;

[0013] The second performance data time series of the target node is determined according to the correlation coefficient corresponding to each adjacent node and the first performance data time series of each adjacent node.

[0014] In an exemplary embodiment of the present disclosure, determining the autocorrelation function of the node according to the second performance data time series includes:

[0015] Processing the second performance data time series using a gated recurrent unit algorithm to obtain a hidden state sequence corresponding to the second performance data time series;

[0016] The autocorrelation function of the node corresponding to the second performance data time series is determined according to the hidden state sequence.

[0017] In an exemplary embodiment of the present disclosure, determining the performance prediction data of the node according to the autocorrelation function and the second performance data time series includes:

[0018] constructing a prediction model based on the autocorrelation function, and training the prediction model using performance data sets at multiple time points in the second performance data time series;

[0019] The performance data corresponding to the most recent time point in the second performance data time series is input into the prediction model, and the output data of the prediction model is set as the performance prediction data of the node.

[0020] In an exemplary embodiment of the present disclosure, determining the performance prediction data of the node according to the autocorrelation function and the second performance data time series further includes:

[0021] Acquire current performance data of the node, and update the first performance data time series and the second performance data time series according to the current performance data;

[0022] The performance data corresponding to the current performance data in the second performance data time series is input into the prediction model, and the output data of the prediction model is set as the performance prediction data of the node.

[0023] According to a second aspect of an embodiment of the present disclosure, there is provided a resource allocation device, including:

[0024] a topology map construction module configured to determine similarities between multiple nodes based on a time series of first performance data of the nodes, and to form a topology map based on nodes whose similarities are greater than a preset value, wherein the nodes are service nodes to be assigned, and the first performance data time series is a sequence of multiple performance data of the service nodes to be assigned that changes over time;

[0025] A time series updating module is configured to determine a correlation coefficient between adjacent nodes in the topology graph, and determine a second performance data time series of each node according to the correlation coefficient;

[0026] An autocorrelation determination module, configured to determine the autocorrelation function of the node according to the second performance data time series;

[0027] a performance prediction module, configured to determine performance prediction data of the node based on the autocorrelation function and the second performance data time series;

[0028] The task node determination module is configured to determine a task node among multiple nodes according to the performance prediction data.

[0029] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the above methods based on instructions stored in the memory.

[0030] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a program is stored. When the program is executed by a processor, the resource allocation method as described in any one of the above items is implemented.

[0031] The embodiment of the present disclosure determines the similarity between the nodes based on the performance data time series between the service nodes to be allocated, forms a topological graph based on the similarity of the service nodes to be allocated with larger similarity, obtains the second performance data time series of each node based on the correlation between the adjacent nodes in the topological graph, so that the second performance data time series can integrate the information of similar nodes; by determining the performance prediction data of the node based on the autocorrelation of the node and the second performance data time series, it is possible to accurately give the performance prediction data by learning the regular characteristics of the node in time and space, and then screen out the service nodes that are more suitable for undertaking cloud computing tasks, namely task nodes, to achieve efficient optimization scheduling of cloud resources and improve cloud resource utilization.

[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0034] Figure 1 is a flow chart of a resource allocation method in an exemplary embodiment of the present disclosure.

[0035] Figure 2 This is a sub-flowchart for determining similarities between nodes in one embodiment of the present disclosure.

[0036] Figure 3 This is a flowchart of determining a second performance data time series in one embodiment of the present disclosure.

[0037] Figure 4 It is a schematic diagram of forming a second performance data time series in an embodiment of the present disclosure.

[0038] Figure 5 It is a schematic diagram of generating performance prediction data in one embodiment of the present disclosure.

[0039] Figure 6 It is a block diagram of a resource allocation device in an exemplary embodiment of the present disclosure.

[0040] Figure 7 is a block diagram of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or that other methods, components, devices, steps, etc. may be employed. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0042] The accompanying drawings are merely schematic illustrations of the present disclosure. Identical reference numerals in the drawings denote identical or similar components, and thus their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0044] Figure 1 is a flow chart of a resource allocation method in an exemplary embodiment of the present disclosure.

[0045] refer to Figure 1 , the resource allocation method 100 may include:

[0046] Step S1: determining similarities between multiple nodes based on a time series of first performance data of the nodes, and forming a topology graph based on nodes having similarities greater than a preset value, wherein the nodes are service nodes to be assigned, and the time series of the first performance data is a sequence of multiple performance data of the service nodes to be assigned that changes over time;

[0047] Step S2, determining a correlation coefficient between adjacent nodes in the topology graph, and determining a second performance data time series of each node according to the correlation coefficient;

[0048] Step S3, determining the autocorrelation function of the node according to the second performance data time series;

[0049] Step S4, determining the performance prediction data of the node according to the autocorrelation function and the second performance data time series;

[0050] Step S5: determining a task node among multiple nodes according to the performance prediction data.

[0051] The embodiment of the present disclosure determines the similarity between the nodes based on the performance data time series between the service nodes to be allocated, forms a topological graph based on the similarity of the service nodes to be allocated with larger similarity, obtains the second performance data time series of each node based on the correlation between the adjacent nodes in the topological graph, so that the second performance data time series can integrate the information of similar nodes; by determining the performance prediction data of the node based on the autocorrelation of the node and the second performance data time series, it is possible to accurately give the performance prediction data by learning the regular characteristics of the node in time and space, and then screen out the service nodes that are more suitable for undertaking cloud computing tasks, namely task nodes, to achieve efficient optimization scheduling of cloud resources and improve cloud resource utilization.

[0052] Next, each step of the resource allocation method 100 is described in detail.

[0053] In step S1, the similarity between multiple nodes is determined based on the first performance data time series of the nodes, and a topological graph is formed based on the nodes whose similarity is greater than a preset value. The nodes are service nodes to be assigned, and the first performance data time series is a sequence of multiple performance data of the service nodes to be assigned that changes over time.

[0054] The nodes in the embodiments of the present disclosure may be referred to as to-be-allocated service nodes. These to-be-allocated service nodes can be either physical nodes or logical nodes used to process tasks in cloud computing. Accordingly, to-be-allocated service nodes correspond to different performance data depending on the type of physical or logical node.

[0055] Exemplarily, the embodiments of the present disclosure can be applied to allocate computing resources for microservices, and the computing resources are allocated based on the corresponding service nodes. The service nodes to be allocated, that is, the service nodes to be allocated, belong to the service nodes that can currently be used for task allocation, and can be updated in real time according to the system resource management situation. The service nodes to be allocated include but are not limited to physical servers, virtual servers, and containers in various forms. These service nodes carry computing resources such as CPU, memory, storage, and network bandwidth for use by microservices. At this time, the performance data of the service nodes to be allocated can be data such as CPU occupancy, memory occupancy, storage occupancy, network bandwidth, etc.

[0056] In some embodiments, the embodiments of the present disclosure may also regard task processing entities such as microservice examples or functional resources as service nodes to be allocated. In a complex microservice system, each microservice instance will call and communicate with each other, and a network structure is formed between them. From the perspective of business scheduling, these microservices can be regarded as a kind of "resource", and different microservices can be allocated to process corresponding user requests according to business scenarios (such as service scenarios for ordinary users and service scenarios for high-security enterprise users). At this time, the performance data of the service nodes to be allocated may include indicators such as call frequency and response time.

[0057] It should be noted that no matter what type of service node to be allocated is, the type of service node to be allocated processed by method 100 is the same.

[0058] For each service node to be assigned, various performance data (e.g., CPU usage, memory usage) can be set for sampling based on its type. These various performance data are sampled at a preset time step. The various performance data samples collected at one time constitute a set of performance data. Different sets of performance data corresponding to different time points constitute a first performance data time series for the service node to be assigned. In other words, the first performance data time series represents how the values ​​of a set of performance data change over time.

[0059] For example, a set of performance data can be generated based on service load-related parameters. Load-related performance data such as CPU usage, memory utilization, and network bandwidth usage can be collected from the service nodes to be assigned at different times. These data are then arranged chronologically to form a first load-related performance data time series. Furthermore, performance data such as the number of requests and service response time can be incorporated into this set of performance data, allowing the first performance data time series to reflect how performance data such as the number of requests and service response time change over time.

[0060] Microservices are typically distributed across multiple physical nodes, and related methods often fail to fully capture the correlations between different physical nodes. Furthermore, the dynamic and short-term nature of microservice resource usage patterns makes it difficult for traditional methods that rely on long-term historical data to accurately understand their short-term characteristics, severely impacting the accuracy of resource scheduling.

[0061] In an embodiment of the present disclosure, after determining the first performance data time series of each node, it is set to determine the similarity between each node based on the first performance data time series of each node, and form a topological graph based on nodes whose similarity is greater than a preset value, so as to consider the impact of similar nodes on the performance of a node and improve the accuracy of performance prediction of the node.

[0062] Figure 2 This is a sub-flowchart for determining similarities between nodes in one embodiment of the present disclosure.

[0063] refer to Figure 2 In one embodiment, step S1 may include:

[0064] Step S11, obtaining a first performance data time series of a first node and a first performance data time series of a second node;

[0065] Step S12, using a dynamic time warping algorithm to calculate a dynamic time warping distance between the first performance data time series of the first node and the first performance data time series of the second node;

[0066] Step S13: determining the similarity between the first node and the second node based on the dynamic regularization distance. The smaller the dynamic regularization distance, the greater the similarity.

[0067] In an exemplary embodiment, the correlation between different service nodes in a cluster can be quantified based on the Dynamic Time Warping (DTW) algorithm. The DTW algorithm can compare the similarity of two time series of performance data of different lengths corresponding to two cloud computing service nodes and find a set of continuous matching relationships that includes all corresponding points in the two series, without being restricted by Euclidean distance.

[0068] The basic principle of DTW is based on distance measurement. For two first performance data time series, for example, a sequence X=(x1, x2, ..., x N ) and a sequence Y of length M = (y1,y2,...,y M ), first, we need to define a distance function to measure the difference between two data points, usually using the Euclidean distance (i.e., the straight-line distance between two points) or other suitable distance metrics. This distance can be called a warp path.

[0069] To find the best matching path between two time series of first-order performance data, DTW constructs a cost matrix. The rows of this matrix correspond to the time points of sequence X, and the columns correspond to the time points of sequence Y. Each element (i, j) in the matrix represents the distance between the i-th point in sequence X and the j-th point in sequence Y (calculated using the distance function defined previously).

[0070] The core of DTW is to find an optimal path in this cost matrix. This path starts from the upper left corner of the matrix (i.e., the first point of sequence X and the first point of sequence Y) and ends at the lower right corner of the matrix (i.e., the last point of sequence X and the last point of sequence Y). The constraints of the path include continuity and monotonicity. Continuity means that adjacent points in the path are also adjacent in the matrix, and there cannot be jump-type connections; monotonicity requires that the path can only move to the right, downward, or to the lower right to ensure the order of the time series. The optimal path is the path that minimizes the sum of the distances corresponding to all points on the path. The sum of the distances corresponding to this path is the DTW distance of the two first performance data time series, which reflects the similarity between the two first performance data time series. The smaller the distance, the more similar the two first performance data time series.

[0071] The DTW algorithm uses the idea of ​​dynamic programming to solve the optimal path and recursively defines a cumulative distance function, which can be expressed as:

[0072] D(i,j)=d(i,j)+min{D(i-1,j), D(i,j-1), D(i-1,j-1)} (1)

[0073] Where D(i, j) is the minimum cumulative distance to point (i, j), and d(i, j) is the distance of point (i, j) in the cost matrix. In this way, the minimum cumulative distance from the starting point to each point can be gradually calculated, and ultimately the minimum cumulative distance from the upper left corner to the lower right corner is found, which is the DTW distance between the two first performance data time series.

[0074] Next, the similarity between the two nodes may be determined based on the DTW distance of the two first performance data time series. The similarity is inversely proportional to the DTW distance. For example, the inverse of the DTW distance may be used as the similarity.

[0075] Since similarity is used to group service nodes to form a topological graph in step S1, in an exemplary embodiment, the service nodes can also be grouped according to the DTW distance. For example, two service nodes whose DTW distance is less than a preset value are divided into one topological graph. Ultimately, the topological graph includes two or more service nodes, and the DTW distances between adjacent service nodes are all less than the preset value.

[0076] When processing node data related to time series, DTW can uncover dynamic similarities in time series. For example, the performance time series of different service nodes may not align perfectly due to different startup times or brief failures. However, DTW can identify similarities in their operating modes, thereby determining whether the service nodes are in similar operating states and capturing dynamic patterns in the time series.

[0077] In addition to using the DTW distance between the first performance data time series to measure the similarity between different service nodes, you can also use Euclidean distance, Manhattan distance, Pearson correlation coefficient, cosine similarity, etc. to measure the similarity between two first performance data time series, and then measure the similarity between two service nodes. These methods may not be as effective as DTW, but they can also reflect the similarity between the operating states of service nodes. Those skilled in the art can determine the method for measuring the similarity between service nodes based on actual conditions.

[0078] When forming a topology graph based on similarity, you first need to set a reasonable similarity threshold (or DTW distance) based on the application scenario and business needs. This threshold will serve as a key criterion for determining whether nodes are connected to each other to form the topology graph. If the threshold is set too low, the connections in the topology graph may be too dense, and the associations between nodes may become unclear, which is not conducive to subsequent analysis and utilization of the graph structure. If the threshold is too high, many nodes that were originally related may become disconnected, resulting in a sparse topology graph and the loss of some important structural information.

[0079] Next, the algorithm traverses each pair of service nodes to be assigned. For any two nodes whose similarity is greater than or equal to the set similarity threshold, an edge connecting the two nodes is added to the topology graph, indicating a relationship between them. As all node pairs are traversed, the resulting topology graph clearly displays the connections between nodes based on similarity.

[0080] It is understandable that, since similarity is determined based on multiple performance data at the same time, multiple service nodes to be assigned may constitute multiple topological graphs according to the similarity between the nodes, and the nodes in different topological graphs are different.

[0081] After initially constructing the topology, you can optimize and verify it based on business domain knowledge and additional data. For example, you can combine historical node interaction records (such as actual call logs between microservices) to check whether the edge connections in the topology are reasonable. If you find that some nodes have a similarity threshold but are not actually directly related, or that they have important connections but are not connected because the similarity is slightly below the threshold, you can adjust the threshold appropriately or manually add / delete some edges to improve the topology structure.

[0082] At the same time, some graph analysis algorithms can also be used to verify some characteristics of the topological map, such as checking whether there are isolated nodes (nodes that are not connected to any other nodes). If there are too many isolated nodes, it may mean that there are unreasonable similarity calculations or threshold settings, and the previous steps need to be re-examined and adjusted; or analyze the connectivity and clustering of the topological map to ensure that the constructed topological map can truly and effectively reflect the actual relationship between nodes, so as to facilitate subsequent in-depth research work such as graph-based clustering analysis and node influence evaluation.

[0083] By forming a topological graph composed of similar nodes, the next step of processing can be performed based on similar nodes.

[0084] In step S2, the correlation coefficient between adjacent nodes in the topology graph is determined, and a second performance data time series of each node is determined based on the correlation coefficient.

[0085] In a topological graph, the adjacent nodes of a node are nodes that have edge connections with it, that is, nodes whose similarity with it is greater than a preset value.

[0086] Figure 3 This is a flowchart of determining a second performance data time series in one embodiment of the present disclosure.

[0087] refer to Figure 3 In an exemplary embodiment, step S2 may include:

[0088] Step S21, determining an attention coefficient of a target node to each adjacent node using a preset loss function and a first performance data time series;

[0089] Step S22: normalize the attention coefficients corresponding to the target node to obtain the correlation coefficients between the target node and each adjacent node;

[0090] Step S23 : determining the second performance data time series of the target node according to the correlation coefficient corresponding to each adjacent node and the first performance data time series of each adjacent node.

[0091] In step S2, it is necessary to measure the influence of each adjacent node on the current node (i.e., the target node). To this end, in the exemplary embodiment, Figure 3The figure shown uses a Graph Attention Network (GAT) for measurement. The Graph Attention Network is a deep learning model based on graph-structured data. Traditional Graph Neural Network (GNN) methods have certain limitations when processing graph data. For example, Graph Convolutional Networks (GCNs) aggregate neighboring node information based on fixed weights, without considering the varying importance of nodes. GAT effectively assigns different attention weights to each node in the graph, thereby better aggregating neighboring node information.

[0092] The following is a detailed introduction based on Figure 3 The method shown is based on GAT to measure the degree of correlation between adjacent nodes, that is, the correlation coefficient.

[0093] In step S21, the attention coefficient between the nodes is first calculated. For node i and its neighbor node j in the graph, the node features are first mapped to a new space through linear transformation. Assuming that the eigenvector of node i is hi and the eigenvector of node j is hj, the vectors Whi and Whj are obtained by linear transformation through the weight matrix W. Among them, in the early stage of model training, the weight matrix W is usually randomly initialized. During the training process of the model, the weight matrix W will be optimized together with other model parameters (such as parameters in the attention function). During the training process, based on the given training data and loss function (such as the cross entropy loss in the node classification task), it is updated by the backpropagation algorithm.

[0094] Then calculate the attention coefficient eij to measure the importance of node j to node i. The calculation formula can be, for example:

[0095] eij=a(Whi,Wh) (2)

[0096] Where a is an attention function, which can be, for example, a simple neural network (such as a single hidden layer feedforward neural network), whose input is the concatenation of vectors Whi and Whj, and whose output is a scalar representing the attention coefficient. Exemplarily, the attention function can be the LeakyReLU activation function.

[0097] In step S22, the calculated attention coefficients alone cannot be directly compared between different nodes, because the numerical ranges of the attention coefficients calculated by different nodes and their neighbors may be inconsistent. Therefore, a normalization function such as the softmax function can be used to perform a normalization operation on all neighbors of the node. The softmax function can convert a set of values ​​into the form of a probability distribution, so that these values ​​are normalized when the sum is 1, which is convenient for comparing the attention levels of different nodes on the same scale. After the attention coefficients are normalized, the correlation coefficient aij between node i and node j is obtained. When aij is large, it means that when updating the state of node i, the features of node j will have a larger weight, which means that the correlation between the two nodes is strong.

[0098] By setting a node i as a target node, the correlation coefficients between the target node and all its adjacent nodes can be calculated. Similarly, the correlation coefficients between all adjacent nodes in the topology graph can be determined.

[0099] In step S23, after the normalized correlation coefficient is obtained, this coefficient can be used to calculate the linear combination of the corresponding features. For example, for a node, the degree of association with each neighboring node (represented by aij) and the features of the neighboring nodes are comprehensively considered, and the node features are updated through linear combination. The final result is used as the new state of each node.

[0100] An example is given below to illustrate step S23.

[0101] Assume that the first performance data time series of node A is hA=[2,3] (the example only represents the characteristic values ​​in two dimensions).

[0102] The first performance data time series of node B is hB=[1,4].

[0103] The first performance data time series of node C is hC=[3,2].

[0104] The first performance data time series of node D is hD=[4,1].

[0105] Assume that after the previous steps (calculating the attention coefficient and normalizing it), the correlation coefficients of node A to nodes B, C, and D are:

[0106] aAB=0.2 (indicates the degree of attention of node A to node B).

[0107] aAC=0.5 (the degree of attention of node A to node C).

[0108] aAD=0.3 (the degree of attention of node A to node D).

[0109] For each neighbor node, multiply its feature vector by the corresponding normalized attention coefficient and then sum them to update the feature of node A and obtain the second performance data time series of node A.

[0110] hA'=aAB*hB+ aAC*hC+ aAD*hD (3)

[0111] Substituting the above values, we get hA' = [0.2, 0.8] + [1.5, 1] ​​+ [1.2, 0.3] = [2.9, 2.1].

[0112] For example, the first performance data time series for node A is [2, 3]. After neighboring nodes are updated, the second performance data time series for node A becomes [2.9, 2.1]. This new time series incorporates information about neighboring nodes and can be useful in subsequent analysis or tasks (such as node classification and predicting a node's influence in a graph).

[0113] In formula (3), only the correlation of nodes B, C, and D to node A is calculated. In some processing methods, the correlation between node A and node A itself can also be calculated at the same time. In this case, the correlation coefficient of node A to node A can be set to 1. Therefore, the final hA' needs to consider the value of the first performance data time series of node A, and the hA' value in formula (3) is updated to [4.9, 5.1].

[0114] How to set the update mode specifically can be set by those skilled in the art according to actual conditions.

[0115] Figure 4 It is a schematic diagram of forming a second performance data time series in an embodiment of the present disclosure.

[0116] refer to Figure 4 For node h1, calculate the correlation coefficients a11~a61 between it and itself and the adjacent nodes h2~h6. Then, based on the correlation coefficients and the first performance data time series hi of each node i (here i=1~6), weightedly obtain the second performance data time series h1' of node h1.

[0117] This approach effectively captures the dependencies between nodes and thus optimizes the scheduling process of cloud resources.

[0118] In general, the input of GAT is a set of node features, h=(h1,h2,...,h N ),h i v R F , where N is the number of nodes and F is the number of features of each node. A new set of node features h′=(h′1,h′2,...,h′N ),h′ i ∈R F , as output.

[0119] For example, the attention coefficient between node i and node j can be calculated by:

[0120] e(h i ,h j )=LeakyReLU(a T ·[Wh i ||Wh j ]) (4)

[0121] where a∈R 2n′ , W∈R n′×n It can be learned, || represents vector concatenation, and LeakyReLU is the activation function.

[0122] In order to make the attention coefficients comparable between different nodes, a softmax function is used for all j∈N i (N i is the set of all neighbors of node i) and normalizes it to get the correlation coefficient aij:

[0123]

[0124] Finally, using the normalized attention coefficient a ij Calculate the linear combination of the corresponding features as the new state of each node:

[0125]

[0126] By considering the attention coefficient between a node and its neighbors and the characteristics of the neighbors, the new state incorporates more information about the node's surrounding environment. This makes the node features richer and more representative, and can better adapt to subsequent tasks such as node classification and link prediction.

[0127] In other embodiments of the present disclosure, other methods may also be used, such as distance-based measurement (such as Euclidean distance, Manhattan distance), similarity coefficient-based measurement (such as cosine similarity, Pearson correlation coefficient), information theory-based measurement (such as mutual information, information gain), etc.

[0128] In step S3, the autocorrelation function of the node is determined according to the second performance data time series.

[0129] In an exemplary embodiment, the second performance data time series can be processed by a gated recurrent unit algorithm to obtain a hidden state sequence corresponding to the second performance data time series, and then the autocorrelation function of the node corresponding to the second performance data time series is determined based on the hidden state sequence.

[0130] Since the GRU (Gated Recurrent Unit) structure is relatively simple, has fewer parameters, and has a faster training speed, in the embodiment of the present disclosure, the GRU model is selected to obtain the autocorrelation function of the node.

[0131] GRU mainly consists of an update gate zt and a reset gate rt.

[0132] Formula (7) can be used to update the hidden state of a certain layer of GRU.

[0133]

[0134] Among them, h t is the hidden state at timestamp t, x t is the input at timestamp t, σ and tanh are activation functions, Wr is the weight matrix of the reset gate, and Wz is the weight matrix of the update gate.

[0135] After using GRU to process the second performance data time series of a node, a hidden state sequence of the node is obtained. The hidden state sequence can be used as the autocorrelation function of the node.

[0136] In addition to using GRU to solve the autocorrelation of nodes and determine the autocorrelation function, the autocorrelation function of the node can also be determined through LSTM (long short-term memory network), ARIMA (autoregressive moving average model), autocorrelation function (ACF), VAR (vector autoregressive model), etc., and technical personnel in this field can make their own choices based on actual conditions.

[0137] In step S4, performance prediction data of the node is determined based on the autocorrelation function and the second performance data time series.

[0138] Figure 5 It is a schematic diagram of generating performance prediction data in one embodiment of the present disclosure.

[0139] refer to Figure 5 In an exemplary embodiment, step S4 may include:

[0140] Step S41, constructing a prediction model based on the autocorrelation function, and using the performance data groups of multiple time points in the second performance data time series to train the prediction model;

[0141] Step S42: input the performance data corresponding to the most recent time point in the second performance data time series into the prediction model, and set the output data of the prediction model as the performance prediction data of the node.

[0142] exist Figure 5 In the illustrated embodiment, after the GRU outputs the hidden state sequence, one or more fully connected layers can be added to construct the prediction layer. The input of the fully connected layer is the hidden state output by the GRU (which can be the hidden state of the last time step, or the entire hidden state sequence fused in some way, such as averaging, concatenation, etc.).

[0143] The prediction model is then trained using supervised learning. A portion of the node's historical performance data (i.e., performance data sets from multiple time points in the second performance data time series) is used as a training set to train the model's parameters (including those in the GRU and the prediction layer), enabling the model to learn the patterns of node performance parameter changes. For example, using mean squared error (MSE) as the loss function, the model parameters are updated through a backpropagation algorithm to minimize the difference between the predicted value and the true value.

[0144] After the prediction model training is completed, the performance data corresponding to the most recent time point in the current second performance data time series can be directly input into the prediction model, processed using the prediction model, and the output data of the prediction model is set as the performance prediction data of the node at the next time point.

[0145] In addition, the current performance data of the node can be obtained, and the first performance data time series and the second performance data time series of the node can be updated according to the current performance data. The performance data corresponding to the current performance data in the second performance data time series is input into the prediction model, and the output data of the prediction model is set as the performance prediction data of the node.

[0146] That is, the current performance data of the node can be collected in real time, and after correlation coefficient processing, the latest set of data in the second performance data time series is formed. The prediction model is used to process the latest set of data to obtain the performance prediction data of the node at the next time point.

[0147] In some embodiments, the above-mentioned GAT process and GRU process can also be combined to form a prediction model, and the prediction model can be jointly trained using the first performance data time series to train the parameters of the prediction model (including parameters in GAT and GRU and parameters of the prediction layer).

[0148] After model training is complete, when new node performance data is input, it will be updated with features by the GAT before being fed into the GRU. The hidden state output by the GRU is transformed by the prediction layer to obtain the node's performance prediction data. For example, performance parameters such as CPU utilization and response time of a node at a certain time step in the future can be predicted.

[0149] In step S5, a task node is determined among multiple nodes according to the performance prediction data.

[0150] At one point in time, after obtaining the performance prediction data for the next point in time based on the latest performance data of each node, tasks can be assigned to each node as needed, and task nodes can be determined among multiple nodes.

[0151] For example, in a cloud computing environment, nodes are numerous server instances. By monitoring and analyzing performance metrics such as CPU usage, memory utilization, and network bandwidth of each server at the current moment, a trained prediction model (such as a model combining GAT and GRU) is used to predict performance for the next time step. If the prediction results show that the CPU utilization of a group of servers will be low in the next period of time, and a new batch of computing tasks (such as large-scale data batch processing tasks) need to be executed at this time, these tasks require high CPU computing power while maintaining relatively stable network bandwidth and memory requirements within an acceptable range, then these tasks can be assigned to the group of servers with low predicted CPU utilization and identified as task nodes.

[0152] For example, in intelligent transportation systems, nodes are traffic light control devices distributed at various intersections. Based on performance data such as the current traffic flow detection data of each signal light node, the current signal light state switching frequency, and the congestion situation on surrounding roads, traffic flow trends at each intersection in the next time period are predicted. If it is predicted that traffic flow at several adjacent intersections will increase significantly in the next few minutes, while some nearby signal light nodes are relatively lightly loaded (i.e., traffic conditions at the currently controlled intersections are relatively smooth and the signal light switching frequency is not high), the task of coordinating traffic flow at these intersections can be assigned to these lightly loaded signal light nodes, making them task nodes. By adjusting the signal light timing plan, traffic flow can be optimized and potential congestion can be alleviated.

[0153] For example, on an industrial production line, nodes are various pieces of production equipment. Sensors collect real-time performance parameters such as operating temperature, speed, and energy consumption, and use this current performance data to predict the equipment's performance in the next production cycle. If it is predicted that the energy consumption of certain pieces of equipment will remain low and their operation will be stable, and there are production tasks that need to be scheduled (such as tasks for producing products with low machining precision but high output) that have certain requirements for the equipment's energy consumption and stability, then these pieces of equipment can be selected as task nodes to undertake these production tasks, thereby improving production efficiency and resource utilization while ensuring the overall stable operation of the production line.

[0154] Table 1 shows the experimental results of node performance prediction using the method provided by the embodiment of the present disclosure. It is obvious that the model using the embodiment of the present disclosure outperforms other models in terms of prediction performance. Traditional FC, MLP, LSTM and GRU models only consider time correlation, while the model of the present disclosure not only considers time correlation, but also takes advantage of topological information, thereby showing superior performance in time series prediction. For example, when the timestamp is 30 seconds, based on MAE data, the prediction accuracy of the model of the present disclosure is improved by 48.27% compared with the RNN (GRU) model that only considers time correlation. Compared with other algorithms, the combination strategy of GAT and GRU in the model of the present disclosure shows significant advantages. These results show that the model of the present disclosure has high accuracy and effectiveness in predicting resource utilization in large-scale cluster environments. By integrating time series data and topological information, resource usage in the cluster can be predicted more accurately, thereby providing cluster operators with more efficient resource management.

[0155] Table 1 Experimental results analysis

[0156]

[0157] In summary, the embodiment of the present disclosure updates node features based on adjacent nodes and obtains performance prediction data for the nodes in combination with the autocorrelation of the nodes, which can greatly improve the accuracy of the performance prediction data and optimize the utilization of cloud computing resources.

[0158] Corresponding to the above method embodiments, the present disclosure also provides a resource allocation device, which can be used to execute the above method embodiments.

[0159] Figure 6 It is a block diagram of a resource allocation device in an exemplary embodiment of the present disclosure.

[0160] refer to Figure 6 , the resource allocation device 600 may include:

[0161] A topology map construction module 61 is configured to determine similarities between multiple nodes based on a time series of first performance data of the nodes, and form a topology map based on nodes whose similarities are greater than a preset value, wherein the nodes are service nodes to be assigned, and the first performance data time series is a sequence of multiple performance data of the service nodes to be assigned that changes over time;

[0162] A time series updating module 62 is configured to determine a correlation coefficient between adjacent nodes in the topology graph, and determine a second performance data time series of each node according to the correlation coefficient;

[0163] An autocorrelation determination module 63 is configured to determine the autocorrelation function of the node according to the second performance data time series;

[0164] A performance prediction module 64 is configured to determine performance prediction data of the node based on the autocorrelation function and the second performance data time series;

[0165] The task node determination module 65 is configured to determine a task node among multiple nodes according to the performance prediction data.

[0166] Since the functions of the apparatus 600 have been described in detail in the corresponding method embodiments, they will not be described in detail herein.

[0167] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0168] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0169] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0170] Refer to the following Figure 7 An electronic device 700 according to this embodiment of the present invention will be described. Figure 7 The electronic device 700 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0171] like Figure 7 As shown, electronic device 700 is implemented as a general-purpose computing device. Components of electronic device 700 may include, but are not limited to, the aforementioned at least one processing unit 710, the aforementioned at least one storage unit 720, and a bus 730 connecting various system components (including storage unit 720 and processing unit 710).

[0172] The storage unit stores program code that can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 710 can perform the method described in the embodiments of the present disclosure.

[0173] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 7201 and / or a cache memory unit 7202 , and may further include a read-only memory unit (ROM) 7203 .

[0174] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0175] Bus 730 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0176] The electronic device 700 can also communicate with one or more external devices 800 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., a router, a modem, etc.). Such communication can occur via an input / output (I / O) interface 750. Furthermore, the electronic device 700 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 700, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0177] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0178] In exemplary embodiments of the present disclosure, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the aforementioned methods of this specification. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the "Exemplary Methods" section of this specification.

[0179] The program product for implementing the above-described method according to an embodiment of the present invention may be a portable compact disc read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0180] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0181] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0182] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0183] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0184] Furthermore, the above-described figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above-described figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0185] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A resource allocation method, characterized in that: include: Determining similarities between multiple nodes based on a time series of first performance data of the nodes, and forming a topology graph based on nodes whose similarities are greater than a preset value, wherein the nodes are service nodes to be assigned, and the first time series of performance data is a sequence of changes over time of multiple performance data of the service nodes to be assigned; Determine a correlation coefficient between adjacent nodes in the topology graph, and determine a second performance data time series of each node according to the correlation coefficient; determining an autocorrelation function of the node based on the second performance data time series; determining performance prediction data of the node according to the autocorrelation function and the second performance data time series; determining a task node among a plurality of nodes according to the performance prediction data; The determining of the correlation coefficient between adjacent nodes in the topology graph and determining the second performance data time series of each node according to the correlation coefficient includes: Determine an attention coefficient of a target node to each adjacent node using a preset loss function and the first performance data time series; Normalizing each attention coefficient corresponding to the target node to obtain a correlation coefficient between the target node and each adjacent node; The second performance data time series of the target node is determined according to the correlation coefficient corresponding to each adjacent node and the first performance data time series of each adjacent node.

2. The resource allocation method according to claim 1, wherein: Determining the similarity between multiple nodes according to the first performance data time series of the nodes includes: Obtain a first performance data time series of the first node and a first performance data time series of the second node; Calculate the dynamic time warping distance between the first performance data time series of the first node and the first performance data time series of the second node using a dynamic time warping algorithm; The similarity between the first node and the second node is determined based on the dynamic time warping distance. The smaller the dynamic time warping distance, the greater the similarity.

3. The resource allocation method according to claim 1, wherein: Determining the autocorrelation function of the node according to the second performance data time series includes: Processing the second performance data time series using a gated recurrent unit algorithm to obtain a hidden state sequence corresponding to the second performance data time series; The autocorrelation function of the node corresponding to the second performance data time series is determined according to the hidden state sequence.

4. The resource allocation method according to claim 1, wherein: Determining the performance prediction data of the node according to the autocorrelation function and the second performance data time series includes: constructing a prediction model based on the autocorrelation function, and training the prediction model using performance data sets at multiple time points in the second performance data time series; The performance data corresponding to the most recent time point in the second performance data time series is input into the prediction model, and the output data of the prediction model is set as the performance prediction data of the node.

5. The resource allocation method according to claim 4, wherein: The determining the performance prediction data of the node according to the autocorrelation function and the second performance data time series further comprises: Acquire current performance data of the node, and update the first performance data time series and the second performance data time series according to the current performance data; The performance data corresponding to the current performance data in the second performance data time series is input into the prediction model, and the output data of the prediction model is set as the performance prediction data of the node.

6. A resource allocation device, characterized in that: include: a topology map construction module configured to determine similarities between multiple nodes based on a time series of first performance data of the nodes, and to form a topology map based on nodes whose similarities are greater than a preset value, wherein the nodes are service nodes to be assigned, and the first performance data time series is a sequence of multiple performance data of the service nodes to be assigned that changes over time; A time series updating module is configured to determine a correlation coefficient between adjacent nodes in the topology graph, and determine a second performance data time series of each node according to the correlation coefficient; An autocorrelation determination module, configured to determine the autocorrelation function of the node according to the second performance data time series; a performance prediction module, configured to determine performance prediction data of the node based on the autocorrelation function and the second performance data time series; a task node determination module, configured to determine a task node among a plurality of nodes according to the performance prediction data; The autocorrelation determination module is configured to: determine the attention coefficient of a target node to each adjacent node through a preset loss function and the first performance data time series; perform standardization on each attention coefficient corresponding to the target node to obtain the correlation coefficient between the target node and each adjacent node; determine the second performance data time series of the target node based on the correlation coefficient corresponding to each adjacent node and the first performance data time series of each adjacent node.

7. An electronic device, characterized in that: include: Memory; as well as A processor coupled to the memory, the processor being configured to execute the method according to any one of claims 1 to 5 based on instructions stored in the memory.

8. A computer-readable storage medium having a program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images