Resource scheduling method, device, equipment, medium and program product

Abstract

The invention provides a resource scheduling method and device, equipment, a medium and a program product, and relates to the technical field of resource scheduling, the method comprises the following steps: constructing a graph data structure according to first resource state data of each network node in a computing power network, the graph data structure comprising a plurality of nodes and edges connecting the nodes, one node corresponds to one network node, and the node is used for indicating first resource state data of the corresponding network node; obtaining a first resource state feature corresponding to the first resource state data of each network node according to a graph convolutional neural network and the graph data structure; according to the first resource state feature of each network node, resource scheduling is executed, and according to the scheme of the application, through the graph convolutional neural network and the constructed graph data structure, the problem of low computing power network resource scheduling efficiency can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource scheduling, and particularly relates to a resource scheduling method, device, equipment, medium and program product BACKGROUND

[0002] In the field of cloud computing, resource scheduling is an important research topic. Existing cloud computing resource scheduling technologies mainly focus on the following aspects: static resource allocation technology, dynamic resource scheduling technology, scheduling technology based on optimization algorithm, and load balancing technology. Among them, the static resource allocation technology usually allocates fixed resources in advance before task execution; the dynamic resource scheduling technology performs resource scheduling based on a preset rule or a prediction model by implementing a monitoring system state; the scheduling technology based on the optimization algorithm uses genetic algorithm, simulated annealing algorithm, and particle swarm optimization algorithm to make resource allocation decisions; the load balancing technology includes static load balancing and dynamic load balancing. The static load balancing allocates tasks based on a predefined strategy and is suitable for scenarios with relatively fixed resource requirements. The dynamic load balancing adjusts the task allocation strategy according to real-time load conditions and needs to rely on current system state feedback for resource adjustment, which cannot achieve long-term optimization.

[0003] However, the above resource scheduling technologies have obvious deficiencies. For example, the static resource allocation technology cannot adapt to real-time load changes, resulting in low resource scheduling efficiency; the dynamic resource allocation technology and the static load balancing have complex and inflexible scheduling rule design, resulting in low resource scheduling efficiency; the dynamic load balancing cannot achieve long-term optimization, resulting in low resource scheduling efficiency; the scheduling technology based on the optimization algorithm has high demand for computing resources and time when processing large-scale cloud computing tasks, and performs poorly in dynamic network environments, resulting in low resource scheduling efficiency. SUMMARY

[0004] At least one embodiment of the present application provides a resource scheduling method, device, equipment, medium and program product, which are used to solve the problem of low efficiency of cloud computing resource scheduling in the prior art.

[0005] To solve the above technical problems, the present application is implemented as follows:

[0006] In a first aspect, the embodiments of the present application provide a resource scheduling method, comprising:

[0007] According to the first resource state data of each network node in the computing power network, a graph data structure is constructed, the graph data structure comprising a plurality of nodes and edges connecting the nodes, one node corresponding to one network node, the node being used to indicate the first resource state data of the corresponding network node;

[0008] According to the graph convolutional neural network and the graph data structure, first resource state features corresponding to first resource state data of each network node are obtained;

[0009] According to the first resource state features of each network node, resource scheduling is performed.

[0010] Optionally, the resource scheduling method, wherein after the resource scheduling is performed, the method further comprises:

[0011] Second resource state data of each network node is obtained;

[0012] According to the long short-term memory network and the second resource state data, second resource state features corresponding to the second resource state data of each network node are obtained;

[0013] According to the second resource state features of each network node, it is determined whether a resource usage amount of the network node exceeds a preset threshold value;

[0014] In a case where the resource usage amount of the network node exceeds the preset threshold value, a warning information is sent, and according to the second resource state features of each network node, resource scheduling is performed.

[0015] Optionally, the resource scheduling method, wherein the first resource state features comprise first-time resource state features and second-time resource state features, and the second time is a time after the first time;

[0016] According to the graph convolutional neural network and the graph data structure, first resource state features corresponding to first resource state data of each network node are obtained, comprising:

[0017] According to the graph convolutional neural network and the graph data structure, first-time resource state features corresponding to first resource state data of each network node are obtained;

[0018] According to the long short-term memory network and the first-time resource state features, second-time resource state features are obtained.

[0019] Optionally, the resource scheduling method, wherein the graph convolutional neural network comprises a convolutional layer, a plurality of hidden layers, and a cross-layer feature aggregation module, wherein,

[0020] The convolutional layer is configured to obtain initial resource state features of each network node according to first resource state data of the network node, and aggregate initial resource state features of first neighbor nodes of the network node into the initial resource state features of the network node to obtain first aggregated state features of the network node for each network node;

[0021] The i-th layer hidden layer is configured to aggregate, for each network node, an initial resource state feature of an (i+1)-th neighbor node of the network node into an i-th aggregated state feature, to obtain an (i+1)-th aggregated state feature of the network node, where i is an integer greater than or equal to 1; when i = 1, the (i+1)-th neighbor node is a second neighbor node, the i-th aggregated state feature is a first aggregated state feature, and the (i+1)-th aggregated state feature is a second aggregated state feature; in the graph data structure, a distance between the second neighbor node and the network node is greater than a distance between the first neighbor node and the network node;

[0022] The cross-layer feature aggregation module is configured to obtain, for each network node, a resource state feature corresponding to the first resource state data of the network node at a first time according to the aggregated state feature of the network node corresponding to each layer hidden layer.

[0023] Optionally, the resource scheduling method, wherein the performing resource scheduling comprises:

[0024] According to the target resource state feature of each network node, the resource interdependence relationship of the network node is obtained, and the target resource state feature includes the first resource state feature or the second resource state feature.

[0025] According to the resource interdependence relationship and the deep Q network, a scheduling strategy is obtained.

[0026] According to the scheduling strategy, a resource scheduling action is performed.

[0027] Optionally, the resource scheduling method, wherein after the performing resource scheduling, the method further comprises:

[0028] According to at least one index, the scheduling strategy is evaluated to obtain a policy evaluation result.

[0029] According to the policy evaluation result, a reward feedback of the deep Q network is obtained.

[0030] According to the reward feedback, the model parameters of the deep Q network are adjusted.

[0031] In a second aspect, the embodiments of the present application further provide a resource scheduling device, comprising:

[0032] A construction module is configured to construct a graph data structure according to the first resource state data of each network node in a computing power network, the graph data structure comprising a plurality of nodes and edges connecting the nodes, one node corresponding to one network node, and the node being used to indicate the first resource state data of the corresponding network node.

[0033] obtaining a first resource state feature corresponding to the first resource state data of each network node according to the graph convolutional neural network and the graph data structure;

[0034] scheduling a resource according to the first resource state feature of each network node.

[0035] In a third aspect, an embodiment of the present application further provides a resource scheduling device, characterized by comprising: a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the processor executes the program or instruction to implement the resource scheduling method in the first aspect.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the resource scheduling method in the first aspect.

[0037] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, and the computer instructions are executed by a processor to implement the resource scheduling method in the first aspect.

[0038] Compared with the prior art, the embodiments of the present application provide a resource scheduling method, device, equipment, medium and program product, and the method comprises: constructing a graph data structure according to first resource state data of each network node in a computing power network, the graph data structure comprises a plurality of nodes and edges connecting the nodes, one node corresponds to one network node, and the node is used to indicate the first resource state data of the corresponding network node; obtaining a first resource state feature corresponding to the first resource state data of each network node according to a graph convolutional neural network and the graph data structure; and scheduling a resource according to the first resource state feature of each network node. In this way, the graph data structure is constructed, the dependence relationship between resources is fully mined through the graph convolutional neural network, global optimization is achieved, and the resource scheduling efficiency is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments, and are not intended to limit the scope of the present application. Moreover, the same reference numerals are used throughout the accompanying drawings to represent same or similar components. In the drawings:

[0040] Figure 1 a flowchart of the resource scheduling method according to an embodiment of the present application;

[0041] Figure 2A flowchart of an embodiment of the resource scheduling method according to the present application;

[0042] Figure 3 An architecture diagram of the graph convolutional neural network according to the present application;

[0043] Figure 4 A flowchart of another embodiment of the resource scheduling method according to the present application;

[0044] Figure 5 An architecture diagram of the long short-term memory network according to the present application;

[0045] Figure 6 An architecture diagram of the application system of the resource scheduling method according to the present application;

[0046] Figure 7 A flowchart of another embodiment of the resource scheduling method according to the present application;

[0047] Figure 8 A structure diagram of the resource scheduling device according to the present application;

[0048] Figure 9 A hardware block diagram of the resource scheduling device according to the present application. DETAILED DESCRIPTION

[0049] The terms "first", "second", and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second" are usually a class, not limited to the number of objects, for example, the first object can be one or more. In addition, "or" in the present application means at least one of the connected objects. For example, "A or B" covers three scenarios, namely, scenario one: including A and not including B; scenario two: including B and not including A; scenario three: including A and B. The character " / " generally indicates that the objects before and after are in an "or" relationship.

[0050] Please refer to Figure 1 The embodiments of the present application provide a resource scheduling method, comprising:

[0051] Step 101, according to the first resource state data of each network node in the computing power network, a graph data structure is constructed, the graph data structure includes a plurality of nodes and edges connecting the nodes, one node corresponds to one network node, and the node is used to indicate the first resource state data of the corresponding network node.

[0052] Exemplarily, the network node comprises a server or a switch.

[0053] Exemplarily, the first resource state data comprises at least one of:

[0054] CPU utilization, used to indicate the processing load of the network node at a specific time step;

[0055] Memory usage, used to indicate the dynamic change of the memory of the network node, helping to predict potential bottlenecks;

[0056] Bandwidth utilization, used to indicate the occupancy of the network node or the connection bandwidth, used to capture network congestion or bandwidth demand changes;

[0057] Network latency, used to identify potential latency risks, especially in high-load situations.

[0058] In some embodiments of the present application, the method further comprises:

[0059] obtaining raw data of the resource state of each network node in the computing power network;

[0060] preprocessing the raw data to obtain the first resource state data.

[0061] Figure 2 A flowchart of one embodiment of the resource scheduling method according to the present application is shown in FIG. 1. As shown in FIG. 1, first, the resource state acquisition unit continuously monitors the resource state of each network node in the computing power network. Specifically, the resource state acquisition unit collects raw data of the resource state through existing monitoring tools such as Prometheus or Zabbix, which includes CPU utilization, memory occupancy, network bandwidth, etc. The resource state acquisition unit communicates with each computing node through an API interface to ensure the real-time and accuracy of data transmission. Figure 2

[0062] It should be noted that the computing node can execute the resource scheduling method according to the present application. Here, in order to cope with the complex data flow in large-scale cloud environment, Kafka message queue can be used for batch data transmission, so as to optimize the real-time and stability of resource scheduling.

[0063] ​Further, the original data is deeply pre-processed to obtain and save the first resource state data. In the pre-processing process, not only the traditional Min-Max normalization algorithm is used to map the data to the [0, 1] interval, but also the wavelet transform and threshold denoising technology are introduced to remove short-term fluctuations and noise in the original data. This multi-level preprocessing ensures the consistency and accuracy of the data, providing high-quality input for the subsequent decision model. The specific formula of the normalization algorithm is as follows:

[0064] ;

[0065] wherein, is the original data, and are the minimum and maximum values of each original data, respectively.

[0066] In some embodiments of the present application, a graph data structure is constructed according to the first resource state data of each network node in the computing power network, comprising:

[0067] According to the graph convolutional neural network and the first resource state data of each network node in the computing power network, a graph data structure is constructed.

[0068] It should be noted that after the first resource state data of each network node in the computing power network is converted into an input vector of the graph convolutional neural network, it is input into the graph convolutional neural network, and the graph data structure is constructed by the graph convolutional neural network. Therefore, compared with the simple vectorization representation method used in the prior art, the graph data structure is used in the embodiments of the present application to deeply mine the complex dependency and influence relationship between resources, thereby providing more informative input for subsequent resource scheduling.

[0069] Step 102, according to the graph convolutional neural network and the graph data structure, obtaining the first resource state feature corresponding to the first resource state data of each network node.

[0070] In one embodiment, the first resource state feature includes a first time resource state feature and a second time resource state feature, wherein the second time is a time after the first time.

[0071] According to the graph convolutional neural network and the graph data structure, obtaining the first resource state feature corresponding to the first resource state data of each network node, comprises:

[0072] According to the graph convolutional neural network and the graph data structure, obtaining the first time resource state feature corresponding to the first resource state data of each network node.

[0073] According to the long short-term memory network and the resource state feature of the first time, the resource state feature of the second time is obtained.

[0074] Here, the first time can be a period of time or a time point. The first time can be understood as the current time.

[0075] In an embodiment, the graph convolutional neural network includes a convolutional layer, a plurality of hidden layers, and a cross-layer feature aggregation module, wherein,

[0076] The convolutional layer is configured to obtain an initial resource state feature of each network node according to first resource state data of the network node, and aggregate the initial resource state features of first neighbor nodes of the network node into the initial resource state feature of the network node to obtain a first aggregated state feature of the network node.

[0077] The i-th hidden layer is configured to aggregate the initial resource state features of (i+1)-th neighbor nodes of each network node into the i-th aggregated state feature to obtain an (i+1)-th aggregated state feature of the network node, where i is an integer greater than or equal to 1; when i = 1, the (i+1)-th neighbor node is a second neighbor node, the i-th aggregated state feature is a first aggregated state feature, and the (i+1)-th aggregated state feature is a second aggregated state feature; in the graph data structure, the distance between the second neighbor node and the network node is greater than the distance between the first neighbor node and the network node.

[0078] The cross-layer feature aggregation module is configured to obtain, for each network node, a resource state feature corresponding to the first resource state data of the network node at the first time according to the aggregated state features of the network node corresponding to each hidden layer.

[0079] Figure 3 An architecture diagram of the graph convolutional neural network is shown in FIG. 1. As shown in FIG. 1, the graph convolutional neural network includes an input layer, a plurality of hidden layers, an aggregation layer, and an output layer. Figure 3

[0080] First, the first resource state data of each network node is input into the input layer of the graph convolutional neural network, and in the input layer, the graph data structure can be constructed. Specifically, in the input layer, the first resource state data of each network node is converted into a graph structure to obtain the graph data structure, wherein each network node corresponds to each node in the graph data structure, and the connection between the network nodes is represented as an edge in the graph data structure. Moreover, each node in the graph data structure can be attached with the first resource state data. ​

[0081] The first resource state data of each network node is first normalized, for example, using Min-Max normalization, to ensure that the data is within the same scale range. In addition, in addition to the first resource state data, the failure rate and task queue length of the network node can be input into the input layer of the graph convolutional neural network; and the network topology structure needs to be input, which is used to indicate the connection between the network nodes.

[0082] The input layer of the graph data structure can construct an adjacency matrix of each network node according to the network topology structure, which is used to indicate the direct connection relationship of the network node, so that the first neighbor node, the second neighbor node, etc. of the network node can be obtained in the convolutional layer of the graph convolutional neural network.

[0083] It should be noted that, in order to enhance the representation ability of the features, the weight value in the adjacency matrix can be dynamically adjusted according to the network state. For example, when the bandwidth utilization between two nodes is high, the connection weight is increased to improve the priority of feature propagation.

[0084] Then, the graph data structure is processed by the convolutional layer, which is used to extract local features in the graph data structure, so that each network node can aggregate the initial resource state features of itself and surrounding neighbor nodes. Specifically, the convolutional layer uses convolution operation at each node: for each network node, the initial resource state feature of the network node is obtained; and for each network node, the initial resource state feature of the first neighbor node of the network node is aggregated into the initial resource state feature of the network node to obtain the first aggregated state feature of the network node.

[0085] Here, the first aggregated state feature needs to introduce nonlinearity through an activation function (such as ReLU) to help the network learn more complex relationships.

[0086] In order to extract more detailed resource state features, the resource scheduling method according to the embodiments of the application can dynamically adjust the weights of different neighbor nodes in the convolution operation at each node by introducing a weight adjustment mechanism based on the node resource utilization, thereby realizing more fine-grained resource state feature extraction.

[0087] Further, the first aggregated state feature of the network node is input into the multi-layer hidden layer of the graph convolutional neural network, each layer of the hidden layer uses convolution operation and activation function, and the multi-layer hidden layer propagates layer by layer. In each layer of the hidden layer, the network node will continuously obtain resource state features from more distant neighbor nodes to capture global features in the graph data structure. This multi-layer feature extraction can understand the complex dependency relationship and features between network node resources at a deeper level.

[0088] The number of layers of the multi-layer hidden layer can correspond to the number of layers of the neighbor nodes, that is, one layer of the hidden layer corresponds to one layer of the neighbor nodes. Alternatively, one layer of the hidden layer corresponds to at least two layers of the neighbor nodes. The neighbor node of the first layer can be understood as the first neighbor node, which is closest to the network node compared with the neighbor nodes of other layers.

[0089] Specifically, taking the number of layers of the multi-layer hidden layer as 5 layers and one layer of the hidden layer corresponding to one layer of the neighbor nodes as an example, the aggregated state features corresponding to each layer of the hidden layer are explained as follows:

[0090] The first layer of the hidden layer: for each network node, the initial resource state features of the second neighbor nodes of the network node are aggregated into the first aggregated state features, to obtain the second aggregated state features of the network node;

[0091] The second layer of the hidden layer: for each network node, the initial resource state features of the third neighbor nodes of the network node are aggregated into the second aggregated state features, to obtain the third aggregated state features of the network node;

[0092] The third layer of the hidden layer: for each network node, the initial resource state features of the fourth neighbor nodes of the network node are aggregated into the third aggregated state features, to obtain the fourth aggregated state features of the network node;

[0093] The fourth layer of the hidden layer: for each network node, the initial resource state features of the fifth neighbor nodes of the network node are aggregated into the fourth aggregated state features, to obtain the fifth aggregated state features of the network node;

[0094] The fifth layer of the hidden layer: for each network node, the initial resource state features of the sixth neighbor nodes of the network node are aggregated into the fifth aggregated state features, to obtain the sixth aggregated state features of the network node.

[0095] Specifically, taking the number of layers of the multi-layer hidden layer as 3 layers and one layer of the hidden layer corresponding to two layers of the neighbor nodes as an example, the aggregated state features corresponding to each layer of the hidden layer are explained as follows:

[0096] The first layer of the hidden layer: for each network node, the initial resource state features of the second neighbor nodes and the third neighbor nodes of the network node are aggregated into the first aggregated state features, to obtain the second aggregated state features of the network node;

[0097] The second layer of the hidden layer: for each network node, the initial resource state features of the fourth neighbor nodes and the fifth neighbor nodes of the network node are aggregated into the second aggregated state features, to obtain the third aggregated state features of the network node;

[0098] The third hidden layer: for each network node, the initial resource state features of the sixth neighbor node and the seventh neighbor node of the network node are aggregated into a third aggregated state feature, and a fourth aggregated state feature of the network node is obtained.

[0099] It should be noted that the first neighbor node, the second neighbor node, the third neighbor node, the fourth neighbor node, the fifth neighbor node, the sixth neighbor node, and the seventh neighbor node are farther and farther away from the network node. That is, the distance between the first neighbor node and the network node is less than the distance between the second neighbor node and the network node, the distance between the second neighbor node and the network node is less than the distance between the third neighbor node and the network node, the distance between the third neighbor node and the network node is less than the distance between the fourth neighbor node and the network node, the distance between the fourth neighbor node and the network node is less than the distance between the fifth neighbor node and the network node, the distance between the fifth neighbor node and the network node is less than the distance between the sixth neighbor node and the network node, and the distance between the sixth neighbor node and the network node is less than the distance between the seventh neighbor node and the network node.

[0100] For each network node, the aggregated state feature of the network node corresponding to each hidden layer is input into a cross-layer feature aggregation module in the graph convolutional neural network, and a resource state feature at a first time corresponding to the first resource state data of the network node is obtained. The cross-layer feature aggregation module makes the aggregated state feature of each layer not only depend on the last hidden layer, but also depend on the aggregated state features of the previous few hidden layers, so as to mine more global resource interdependence relationships. The cross-layer feature aggregation module helps to enhance the perception of long-distance node dependence, and can identify more potential resource dependence features, especially in complex network topologies.

[0101] Further, the graph convolutional neural network further includes an aggregation layer, which is configured to perform graph aggregation on the resource state features of the network nodes in the computing power network at the first time. The aggregation layer uses a graph aggregation operation to integrate the first resource state features of the network nodes at the first time, so as to make further global decisions. The aggregation method can include various methods, such as maximum pooling (Mac) or average pooling (Average Pooling). For example, the resource scheduling method described in the embodiments of the present application can use maximum pooling for aggregation to obtain global features of the entire computing power network.

[0102] In addition, the resource scheduling method described in the embodiments of the present application also includes a dynamic graph aggregation strategy, which is used to indicate that the maximum pooling or average pooling is dynamically selected according to the network state. For example, when the network is in a high load state, the maximum pooling is preferentially selected to highlight the utilization state of the key resources.

[0103] Here, through the graph convolution operation, each network node aggregates the initial resource state features of the neighbor nodes into its own initial resource state features. Specifically, for the feature update of a network node v, the formula is as follows:

[0104] ;

[0105] wherein, denotes the neighbor node set of the network node , is a normalization constant, is the weight matrix of the layer, is the initial resource state feature of the network node in the layer, is a nonlinear activation function (such as ReLU).

[0106] Through the above graph convolution operation, each node aggregates the initial resource features of itself and the neighbor nodes, captures the dependency relationship between resources, and provides rich structural information for subsequent scheduling decisions.

[0107] Next, the long short-term memory network is input with the resource state feature of the first time corresponding to the first resource state data of each network node obtained by the graph convolutional neural network. The resource state feature of the first time includes the local and overall resource state features of each network node, which is used to represent the resource state of the network node in the computing power network, and is essentially sequence data changing with time.

[0108] The long short-term memory network generates the resource state feature of the second time by using its memory and processing capability for time series.

[0109] After the processing of the graph convolutional neural network is completed, the vector hv corresponding to the resource state feature of the first time of each network node represents the resource state information of the network node at the first time. These resource state features are serialized into time series data to form a sequence {hv(t), hv(t-1),...} composed of network node features at different time steps.

[0110] Therefore, the vector corresponding to the resource state feature of the first time output by the graph convolutional neural network is input into the long short-term memory network in the order of time sequence, and the long short-term memory network can capture the dynamic change pattern of the resource state by analyzing the features of the historical time steps.

[0111] In the embodiments of the present application, the long short-term memory network is used for time series prediction, and the input layer receives the output from the graph convolutional neural network, which represents the resource utilization of the network nodes (such as the sequence data of CPU, memory, etc. changing over time).

[0112] The long short-term memory network includes a memory mechanism, and the key lies in its gating mechanism, which can effectively handle long-term dependencies. The forget gate, input gate and output gate in the long short-term memory network control the input sequence, remember important historical information, forget irrelevant information, and generate new state information.

[0113] Forget gate: decides which historical information to discard.

[0114] Input gate: updates the input information of the current time step.

[0115] Output gate: generates the hidden state of the next time step, representing the information of the current time step.

[0116] The long short-term memory network can be used for load prediction, and the output represents the second resource state feature of the future time. According to these prediction values, the system can judge the trend of resource demand change in the next few minutes, and perform resource allocation or scheduling in advance to avoid resource bottlenecks.

[0117] The resource scheduling method in the embodiments of the present application directly combines the graph convolutional neural network and the long short-term memory network to construct an adaptive and dynamic resource scheduling framework, which can realize more refined scheduling under complex network topology and uncertain resource state.

[0118] Step 103, performing resource scheduling according to the first resource state feature of each network node.

[0119] In one embodiment, the method further comprises, after performing resource scheduling:

[0120] Obtaining second resource state data of each network node;

[0121] Obtaining a second resource state feature corresponding to the second resource state data of each network node according to the long short-term memory network and the second resource state data;

[0122] determine whether the resource usage of each network node exceeds a preset threshold according to the second resource state feature of the network node;

[0123] In the case where the resource usage of the network node exceeds the preset threshold, send a warning message, and perform resource scheduling according to the second resource state feature of each network node.

[0124] In some embodiments of the present application, the long short-term memory network can be used for anomaly detection. The long short-term memory network processes second resource state data belonging to a time series, and the model can learn the normal resource usage pattern and predict the future change of the resource state. When the long short-term memory network predicts that the resource usage of a certain network node will exceed the preset threshold or change abnormally, the system can give an early warning and trigger an emergency scheduling strategy.

[0125] When an abnormal situation is detected, Figure 6 the anomaly handling module in the abnormal situation triggers an emergency strategy to adjust resource scheduling and ensure stability.

[0126] In terms of abnormal situation handling mechanism, the long short-term memory network is introduced for deep anomaly detection, replacing the traditional simple anomaly detection method based on threshold. The long short-term memory network can predict potential abnormal situations based on complex patterns in the current resource state data. In a specific technical implementation, the long short-term memory network continuously monitors the usage of resources such as CPU utilization and network latency. When the long short-term memory network predicts that the resource usage pattern will be abnormal, it can respond in advance and take appropriate emergency measures,

[0127] such as reassigning tasks or starting backup nodes. This prediction-based anomaly handling method greatly improves the robustness and response speed of the system. After the decision module, if an abnormal situation occurs, the system stability is improved through the abnormal situation handling mechanism, and the technical solution is as shown in Figure 4 .

[0128] The embodiments of the present application directly combine the graph convolutional neural network and the long short-term memory network. The graph convolutional neural network is used to process complex network topology information, which can fully utilize the network structure information in resource scheduling decision-making and capture the resource interdependence relationship of network nodes. The output is then transmitted to the long short-term memory network to form a closer combination, further improving the processing capability of the graph data structure, especially in processing complex graph structure data and anomaly detection.

[0129] In terms of emergency strategy execution, the embodiments of the present application can be deeply integrated with resource scheduling systems such as Kubernetes or OpenStack, and can quickly respond to abnormal detection results. Specifically, when it is detected that the network delay of a certain network node is too high, the task will be immediately migrated to a node with lower delay. The Pod migration function of Kubernetes plays an important role here, and the embodiments of the present application can dynamically control the scheduling of Pods through API to ensure that the task is always executed on the optimal network node.

[0130] To further improve the global optimization capability, the embodiments of the present application also introduce a load prediction mechanism. The load prediction module implemented based on the long short-term memory network can predict the load trend in the next few minutes in advance before resource scheduling. In terms of technical implementation, the long short-term memory network trains the model through historical resource state data to capture the rules of resource usage, and predicts future load fluctuations based on this, as shown in Figure 4 and Figure 5 When it is predicted that the future load will increase significantly, the resource allocation of the corresponding network node can be increased in advance or more network nodes can be started. This predictive scheduling greatly improves the overall scheduling efficiency and stability, and can effectively avoid the collapse caused by resource bottlenecks or sudden load growth.

[0131] Figure 5 A typical long short-term memory network architecture for time series prediction is shown, including an input layer (inputting second resource state data), an LSTM layer, a hidden layer (feature mapping), and an output layer (outputting second resource state features). The following is the application process of the long short-term memory network:

[0132] Step 1: Input layer, input second resource state data;

[0133] Data collection: First, the second resource state data needs to be collected as the input of the model. These data can include network node resource usage, CPU load, bandwidth usage, and other time-varying indicators.

[0134] Data preprocessing: The collected time series data is standardized or normalized to ensure that the data is within the same range, facilitating model processing.

[0135] Step 2: LSTM layer, time series modeling;

[0136] Time series feature extraction: The design of the LSTM layer aims to capture the long-term dependencies of time series data. At each time step, LSTM reads the current input load data, while combining the previous hidden state and cell state , and through the input gate , forget gate and output gate to update cell state and hidden state . decide how much hidden state information to output to the next layer, decide which current information to update, decide how much hidden state information to output to the next layer. LSTM can effectively remember important historical information while forgetting unimportant information through its gating mechanism, so it can handle time series data with long-term dependencies well.

[0137] Step 3: Hidden layer, feature mapping;

[0138] After processing by the LSTM layer, the data is passed to the hidden layer, which is used to further nonlinearly map the features extracted by LSTM. This layer can be composed of one or more fully connected layers to further learn and combine feature representations at different time steps. The purpose of this step is to convert complex temporal dependencies into higher-level feature representations, making it easier for the final prediction layer to make more accurate future load fluctuation predictions.

[0139] Step 4: Output layer, output second resource state feature, for load fluctuation prediction;

[0140] Predict future load fluctuations: The features mapped by the hidden layer will be passed to the output layer. In the output layer, the model finally generates a future load fluctuation prediction value. This value represents the resource load situation in the network or system at a specific future time point, such as CPU load, memory usage, or bandwidth demand, etc. The output can be a scalar (such as a single time step load prediction), or a vector (such as multiple future time step load fluctuation predictions), depending on the needs of the application scenario.

[0141] Step 5: Model training and optimization;

[0142] The mean squared error (MSE) is used as the loss function in the embodiments of the present application to measure the difference between the predicted load and the actual load. Through the backpropagation algorithm (Backpropagation Through Time, BPTT), the parameters in the long short-term memory network are updated to minimize the loss function, gradually improving the model's ability to predict load fluctuations.

[0143] The application scenarios of the future load fluctuation prediction technology include:

[0144] Cloud computing resource scheduling: In a cloud computing environment, using long short-term memory networks to predict future load fluctuations can help anticipate potential resource bottlenecks in network nodes, allowing for reasonable adjustments to resource allocation strategies;

[0145] Data center dynamic expansion: by predicting future resource load, dynamic expansion and contraction of resources can be realized, avoiding overuse or idle of resources;

[0146] Network traffic prediction: can be used for network traffic prediction, to adjust the routing strategy in advance to prevent traffic congestion.

[0147] In one embodiment, the resource scheduling includes:

[0148] According to the target resource state feature of each network node, the resource interdependence relationship of the network node is obtained, and the target resource state feature includes the first resource state feature or the second resource state feature.

[0149] According to the resource interdependence relationship and the deep Q network, a scheduling strategy is obtained.

[0150] According to the scheduling strategy, a resource scheduling action is performed.

[0151] In some embodiments of the present application, the target resource state feature is used to analyze the resource interdependence relationship of the network node. Through feature extraction and aggregation, it can be clearly understood which network node has strong resource interdependence and which network node is in a bottleneck state. At this time, the scheduling algorithm can use this information to make scheduling decisions, obtain a scheduling strategy, and select how to allocate resources and schedule tasks in the network. The network node bottleneck detection can be performed by combining the multi-layer feature extraction result of the graph convolutional neural network and the special needs in the network resource scheduling scenario, mapping the resource state feature output by the graph convolutional neural network to the resource dependence matrix, accurately positioning which network node is the bottleneck of the system, and predicting possible resource conflicts.

[0152] The deep Q network used in the embodiments of the present application is essentially a reinforcement learning model, which can select the optimal scheduling action according to the resource interdependence relationship, Figure 6 The action policy unit in the above formula is responsible for evaluating the value of different actions and making a choice. The resource scheduling execution unit adjusts the resource allocation and task scheduling according to the output of the action selection unit, and the specific process is as follows:

[0153] 1、The application embodiment described resource scheduling method application system, abbreviated as resource scheduling system, hereinafter can also be referred to as system. According to the feature extracted by the graph convolutional neural network and the analyzed inter-resource dependency, the optimal resource strategy is made. It can adjust the allocation of resources according to the load and state of different nodes in the network, such as increasing or decreasing bandwidth, allocating more computing resources or adjusting the priority of network nodes. Therefore, the embodiment of the application essentially adopts a joint optimization mechanism based on reinforcement learning and graph convolutional neural network. The graph convolutional neural network extracts resource state features, and the deep Q network estimates the next resource scheduling action according to the current resource state features, so that the system can form a better scheduling strategy before actual execution.

[0154] 2、Execution scheduling and feedback

[0155] The system executes specific resource scheduling actions in the computing power network according to the scheduling strategy, such as reallocating bandwidth, adjusting network topology or optimizing routing, etc. After the execution of resource scheduling actions, the system will continue to monitor the network state and constantly optimize the scheduling strategy through the feedback mechanism to ensure efficient use of network resources.

[0156] Figure 6 The reinforcement learning module in the above formula is based on deep Q network and policy gradient algorithm, combined with adaptive mechanism to handle dynamic resource scheduling problems. In specific implementation, the system first estimates the Q value of each possible resource scheduling action through the deep Q network, and then selects the optimal action based on the Q value. Unlike traditional Q-learning, the deep Q network estimates the Q value through a neural network, which can handle complex multi-dimensional input data, and uses an ε-greedy strategy to balance exploration and utilization. The adaptive policy gradient algorithm in the system dynamically adjusts the model parameters according to the current environment and load conditions. For example, in high load conditions, the system will tend to choose a more conservative scheduling strategy to ensure system stability; while in the case of light load, the system will increase the exploration of new scheduling strategies.

[0157] It should be noted that if the target resource state feature is the second resource state feature, it can generate more reasonable scheduling decisions in combination with the previous resource scheduling history. The LSTM model can propose abnormal handling measures when making decisions according to abnormal conditions in the scheduling process (such as node resource depletion, traffic bottlenecks, etc.).

[0158] Step 3, decision execution and feedback

[0159] The scheduling strategy generated by the long short-term memory network is executed, and resources are re-allocated according to the scheduling strategy. At this time, the tasks in the computing power network will be executed according to the optimized path or resource distribution. The execution result is fed back to the system through the monitoring mechanism, and the long short-term memory network will continuously adjust the future decision according to the feedback to achieve the global optimization.

[0160] In one embodiment, optionally, after executing the resource scheduling, the method further comprises:

[0161] According to at least one index, the scheduling strategy is evaluated to obtain a strategy evaluation result;

[0162] According to the strategy evaluation result, a reward feedback of the deep Q network is obtained;

[0163] According to the reward feedback, the model parameters of the deep Q network are adjusted.

[0164] Next, the execution process of the long short-term memory network described in the embodiments of the present application will be specifically described in combination with Figure 4 and Figure 5 , including the following steps:

[0165] Step 1: input layer, used for collecting and preprocessing resource state data;

[0166] (1) Data collection

[0167] Input parameters: the input of the model includes resource state data;

[0168] Sampling frequency: in order to capture the change trend of resource load, the data sampling frequency is set to sample once per minute to balance the data volume and real-time performance.

[0169] (2) Data preprocessing

[0170] Abnormal processing: eliminate or smooth obvious outliers to reduce the interference of noise on the model and improve the prediction accuracy.

[0171] Step 2: LSTM layer, used for time series modeling;

[0172] (1) Time series feature extraction

[0173] Input gate, forget gate, output gate: the gating mechanism of LSTM is used to selectively remember or forget the resource state data at each time step to adapt to the change of load fluctuation. Specifically:

[0174] Input gate: pass the current resource state data (such as CPU, memory, etc.) to the cell state to control which information is written;

[0175] Forget gate: selectively discard unimportant resource state data, such as short-term abnormal fluctuations, to reduce the impact of noise on prediction;

[0176] Output gate: determine the amount of information output at the current time step, retain information that contributes to future load prediction.

[0177] Hidden state and cell state: LSTM maintains a hidden state and cell state at each time step, representing the system state at the current time, and is constantly updated at each time to ensure that long-term dependencies of resource load are captured.

[0178] (2) Time series sliding window:

[0179] To capture long-term trends in load, the model uses a sliding window approach, using the data from the previous T time steps to predict the resource state at T+1 time step. The window length T is selected based on experimental adjustment, generally set to 10 to 20 time steps, depending on the task requirements and the rate of change of resource load.

[0180] Step 3: Hidden layer, for feature mapping;

[0181] (1) Hidden layer design

[0182] After the LSTM layer output, 1 to 2 fully connected hidden layers are added to further learn the complex nonlinear relationships in the time series. The dimension of the hidden layer is set according to the actual demand, such as 64 or 128 units;

[0183] Activation function: The hidden layer can apply the ReLU activation function to map the features to a higher-dimensional space, improving the model's expression ability. Nonlinear mapping helps the model extract features directly related to future load from historical load.

[0184] (2) Feature dimensionality reduction and combination

[0185] The hidden layer can filter out more representative features through feature selection or dimensionality reduction techniques to avoid overfitting. The basis for feature selection is the correlation between the changes in different resource parameters and future load.

[0186] Step 4: Output layer, for future load fluctuation prediction, output the second resource state feature;

[0187] (1) Prediction output

[0188] Output type: The output layer generates a prediction of future load, which can be single-step prediction (such as the CPU utilization of the next time step) or multi-step prediction (such as the load sequence in the next 10 minutes); The prediction dimension specifically includes CPU load, memory usage, bandwidth demand, etc.

[0189] Prediction format: Depending on business needs, scalar output or vector output (multi-step prediction) can be used. Scalar output is suitable for short-term scheduling decisions, while vector output is more suitable for long-term scheduling planning.

[0190] (2) Threshold setting and early warning

[0191] The system can set thresholds for different resource parameters. For example, when CPU utilization exceeds 80% or network latency exceeds a certain threshold, the system will issue an early warning signal so that measures can be taken in advance to reallocate resources or migrate tasks.

[0192] Step 5: Model training and optimization;

[0193] After the above steps are completed, training and optimization can begin. In this embodiment, mean squared error can be selected as the loss function. During the model optimization process, the learning rate is adaptively adjusted according to the training progress of the model. The Adam optimizer or learning rate decay technique is used to ensure that the model can converge quickly and avoid overfitting.

[0194] The system evaluates the effectiveness of scheduling actions, calculates corresponding reward feedback, and feeds this feedback to the deep Q-network via a reward feedback module. Based on the reward feedback, the deep Q-network adjusts its model parameters to optimize future decisions.

[0195] The resource scheduling method described in this application introduces a multi-dimensional reward mechanism. Specifically, this application evaluates the scheduling strategy based on multiple indicators such as task processing efficiency, resource consumption, and load balancing. For example, if the system reduces resource consumption while ensuring task processing efficiency, it will provide positive reward feedback; if uneven task allocation leads to excessive load on some network nodes, the system will provide negative reward feedback. This multi-dimensional reward mechanism provides the system with more detailed optimization directions, ensuring that the decision model can achieve optimal scheduling results under different conditions.

[0196] The entire system iterates through continuous reinforcement learning. Figure 6The policy update module optimizes the scheduling policy based on the latest reward feedback. In its implementation, an adaptive policy gradient algorithm is used. This algorithm dynamically adjusts the model parameters of the deep Q-network based on the current reward feedback, resulting in faster model convergence and the ability to quickly respond to changes in the computing power network. The reinforcement learning module optimizes the scheduling policy through continuous interaction between the agent and the environment. The system observes the environmental state, makes actions (i.e., scheduling decisions), and continuously adjusts the scheduling policy based on the reward feedback received to optimize resource scheduling. In this process, the policy gradient algorithm is the core; it updates the scheduling policy based on reward feedback, enabling the agent to make better scheduling decisions in complex computing power networks. The specific process includes the following steps:

[0197] Initialize the policy network parameters and set the initial learning rate.

[0198] The system monitors the status and inputs resource status characteristics and task information into the deep Q network.

[0199] Generate resource scheduling actions and select the optimal scheduling strategy based on the action distribution output by the deep Q-network.

[0200] Perform resource scheduling actions and monitor system feedback in real time.

[0201] Update the scheduling strategy and adjust the model parameters of the deep Q-network based on the reward feedback obtained after scheduling, including:

[0202] (1) Positive rewards increase the probability of choosing the corresponding action;

[0203] (2) Negative rewards reduce the probability of choosing an action.

[0204] Adaptively adjust the learning rate based on reward feedback to respond more quickly to environmental changes.

[0205] Iterate continuously and improve the effectiveness of the scheduling strategy through interactive learning.

[0206] Therefore, this application's embodiments achieve a significant technological breakthrough in the field of resource scheduling for computing power networks by introducing graph convolutional networks for resource state modeling, employing anomaly detection and load prediction mechanisms based on long short-term memory networks, and combining deep Q-networks and adaptive policy gradient algorithms. This not only enables the system to intelligently adapt to complex resource usage patterns but also optimizes resource utilization and task processing efficiency through multi-dimensional feedback mechanisms. The entire system possesses high dynamic response capabilities and global optimization capabilities, significantly improving the resource scheduling efficiency and system stability of computing power networks in practical applications.

[0207] To better understand the resource scheduling method described in the embodiments of this application, the following is combined with... Figure 6 and Figure 7 Let's illustrate with examples.

[0208] In a computing network, multiple tasks run simultaneously, requiring continuous resource monitoring and scheduling to ensure efficient and stable task execution. This includes how to schedule resources when the load suddenly increases, and how to optimize scheduling strategies using reinforcement learning algorithms.

[0209] (1) Resource status monitoring and data acquisition:

[0210] In a computing network, the system's resource monitoring and scheduling module continuously monitors the resource utilization of each network node. When CPU utilization suddenly increases and memory approaches saturation, the resource status acquisition unit promptly collects relevant data. This data may include: CPU utilization, memory usage, task queue length, disk I / O, network bandwidth, etc., reflecting the current pressure status of the node.

[0211] (2) State representation and vectorization:

[0212] The collected raw data is transmitted to the state representation module, which preprocesses the data, including noise removal and outlier detection. Subsequently, this processed data is vectorized to generate multidimensional feature vectors suitable for model input. For example, the resource state of each network node can be transformed into a high-dimensional vector containing multiple resource usage metrics such as CPU, memory, and I / O.

[0213] In this step, graph convolutional neural networks and long short-term memory networks can be used to model and learn complex resource dependencies, extract global or temporal features, and capture the dynamic relationships between resources among network nodes in the system.

[0214] (3) Reinforcement learning model decision-making;

[0215] The processed resource status features are then input into the reinforcement learning model. Upon receiving the current resource status, the reinforcement learning model evaluates the current resource situation based on existing scheduling policies and determines whether resource scheduling actions are necessary. If the reinforcement learning model determines that the load on network nodes has exceeded the system's safety threshold, it will decide to take resource scheduling action.

[0216] Action selection:

[0217] The model selects a series of scheduling actions through a decision-making process. For example, under conditions of high load, the model might choose to migrate some tasks to other idle network nodes, thereby reducing the resource pressure on the current network nodes. Reinforcement learning models select the optimal scheduling scheme through the probability distribution generated by the policy network, for example:

[0218] Which tasks will be migrated?

[0219] To which network node to migrate;

[0220] Adjusting resource allocation strategies, etc.

[0221] (4) Resource scheduling execution:

[0222] The resource scheduling execution unit executes specific scheduling actions based on the decisions of the reinforcement learning model. For example, the system may select some heavily loaded tasks to migrate to other less loaded network nodes. During the scheduling process, the system intelligently selects the tasks to be migrated and the target network nodes to ensure load balancing while avoiding the creation of new performance bottlenecks.

[0223] (5) Evaluation and feedback of scheduling effectiveness:

[0224] After the resource scheduling action is completed, the system will monitor resource usage again, especially CPU utilization and memory usage after task migration. At this point, the system may find that the scheduling effect is significant: CPU utilization and memory usage have been alleviated, task processing efficiency remains stable, and there are no obvious task delays or failures. Based on the effect of the scheduling action, the reward feedback module will calculate the corresponding reward value. If the scheduling strategy successfully reduces the system load and task execution is not affected, the reward feedback will be relatively high; conversely, if the scheduling leads to a decrease in system performance or task failure, the reward feedback will be reduced accordingly.

[0225] (6) Strategy updates and optimizations:

[0226] After the reward is fed back to the reinforcement learning model, the model updates its scheduling policy based on the reward feedback. This process is implemented through a policy gradient algorithm, allowing the model to continuously learn from past experiences and optimize future decisions. After each policy update, the model will gradually be able to make better scheduling strategies under similar resource constraints, reducing the risk of overload.

[0227] (7) Exception handling mechanism:

[0228] If, during scheduling, the system detects a failure in a network node (e.g., a network node crash or network interruption), the anomaly handling mechanism module immediately activates emergency strategies. These strategies include reassigning tasks currently executing on that network node to other healthy network nodes. This allows the system to respond quickly to failures, ensuring tasks continue to run normally and guaranteeing high availability in the cloud environment. During this process, the system also monitors the effectiveness of anomaly handling and provides corresponding feedback to the reinforcement learning model based on the results. For example, if the system successfully mitigates the impact of a failure by quickly mitigating tasks, the model receives a positive reward and will be more likely to make correct decisions in similar situations in the future.

[0229] In summary, the resource scheduling method described in this application essentially provides a reinforcement learning-based optimization and improvement technology for cloud computing network resource scheduling. By introducing reinforcement learning models and multiple optimization mechanisms, it achieves intelligent scheduling and optimization of computing network resources. This application's embodiments can comprehensively optimize cloud computing resource scheduling, utilizing reinforcement learning, deep learning, and multi-dimensional optimization mechanisms to intelligently handle complex cloud computing environments. From resource acquisition, data modeling, decision selection to anomaly detection and global optimization, it breaks away from traditional modular design, constructing a more intelligent and flexible resource scheduling framework. Furthermore, it can shorten the model training cycle and improve the real-time performance and adaptability of the scheduling system.

[0230] Specifically, the resource scheduling method described in the embodiments of this application has the following beneficial effects:

[0231] The resource scheduling is more intelligent and adaptive: Employing reinforcement learning algorithms, the embodiments of this application can more intelligently and autonomously learn and dynamically adjust resource scheduling strategies based on real-time status and historical experience in the cloud computing environment. Compared with existing technologies, it can better adapt to the complex and ever-changing cloud computing environment, avoiding the complexity and limitations of rule design. For example, when faced with sudden high-load tasks, it can quickly adjust resource allocation and improve task processing efficiency.

[0232] Significantly shortened model training cycle: By employing an advanced policy update algorithm, the embodiments of this application significantly reduce the amount of data and computing resources required for model training, enabling the model to converge faster and adapt to dynamic environments. Compared to the lengthy training cycle in existing technologies, this proposal significantly improves the real-time performance of the system, allowing resource scheduling decisions to respond more quickly to environmental changes.

[0233] Improving resource utilization and system performance: Leveraging the adaptive optimization capabilities and multi-dimensional reward feedback mechanisms of reinforcement learning models, the embodiments of this application can achieve more precise comprehensive optimization in multiple aspects such as task processing efficiency, resource consumption, and load balancing. For example, by intelligently adjusting resource allocation, it ensures high resource utilization under different load conditions, avoiding resource waste or performance degradation, thereby effectively improving system performance.

[0234] Enhancing the system's responsiveness to emergencies: This application's embodiments integrate anomaly handling mechanisms into the reinforcement learning model, enabling faster detection of anomalies such as node failures, network latency, and sudden load spikes, and timely adjustment of scheduling strategies. Compared to existing technologies, this more effectively ensures the system's stability and reliability under abnormal conditions, reducing the risk of system performance degradation and service interruptions caused by anomalies.

[0235] It has global optimization capabilities: Unlike existing scheduling schemes based on rules or local optimization, the embodiments of this application, through the global policy update mechanism of reinforcement learning algorithm, combined with the collaborative work of resource monitoring and scheduling modules, can achieve global optimal allocation of resources, better balance the overall load, reduce the occurrence of system bottlenecks, and thus improve the overall operating efficiency of the cloud computing environment.

[0236] Taking into account multiple optimization objectives: Existing technologies typically optimize only a single objective, while the embodiments of this application achieve a more comprehensive optimization effect by comprehensively considering multiple indicators such as task processing efficiency, resource consumption, load balancing, and service quality through a multi-dimensional reward and feedback mechanism. For example, when scheduling resources, it not only focuses on task processing time but also takes into account resource consumption and load balancing to achieve the overall optimal scheduling decision.

[0237] Reduced system resource overhead: By optimizing the algorithm and designing a lightweight model, the embodiments of this application significantly reduce system resource overhead. While ensuring scheduling accuracy, the computational burden is reduced, enabling the system to operate efficiently while maintaining low resource consumption. Compared to the high resource consumption of reinforcement learning models in the prior art, the embodiments of this application can effectively improve the overall resource utilization of the cloud computing platform.

[0238] Please refer to Figure 8 This application also provides a resource scheduling device, including:

[0239] The construction module 801 is used to construct a graph data structure based on the first resource status data of each network node in the computing power network. The graph data structure includes multiple nodes and edges connecting the nodes. Each node corresponds to a network node, and the node is used to indicate the first resource status data of the corresponding network node.

[0240] The obtaining module 802 is used to obtain a first resource state feature corresponding to the first resource state data of each network node based on the graph convolutional neural network and the graph data structure.

[0241] The scheduling module 803 is used to perform resource scheduling based on the first resource status characteristics of each network node.

[0242] Optionally, the resource scheduling device further includes:

[0243] The acquisition module is used to acquire the second resource status data of each of the network nodes;

[0244] The prediction module is used to obtain the second resource state features corresponding to the second resource state data of each network node based on the long short-term memory network and the second resource state data;

[0245] The judgment module is used to determine whether the resource usage of the network node exceeds a preset threshold based on the second resource status characteristics of each network node.

[0246] The execution module is used to send an early warning message when the resource usage of the network node exceeds a preset threshold, and to perform resource scheduling based on the second resource status characteristics of each network node.

[0247] Optionally, in the resource scheduling device, the first resource status feature includes a resource status feature at a first time and a resource status feature at a second time, wherein the second time is a time after the first time.

[0248] The obtaining module 802 is specifically used for:

[0249] Based on the graph convolutional neural network and the graph data structure, the resource state features corresponding to the first time period of the first resource state data of each network node are obtained;

[0250] Based on the Long Short-Term Memory network and the resource state characteristics at the first time, the resource state characteristics at the second time are obtained.

[0251] Optionally, in the resource scheduling device, the graph convolutional neural network includes convolutional layers, multiple hidden layers, and a cross-layer feature aggregation module.

[0252] The convolutional layer is used to obtain the initial resource state features of the network node based on the first resource state data of each network node, and for each network node, to aggregate the initial resource state features of the first neighboring nodes of the network node into the initial resource state features of the network node to obtain the first aggregated state features of the network node.

[0253] The i-th hidden layer is used to aggregate the initial resource state features of the (i+1)-th neighbor nodes of each network node into the i-th aggregated state feature to obtain the (i+1)-th aggregated state feature of the network node; where i is an integer greater than or equal to 1; when i=1, the (i+1)-th neighbor node is the second neighbor node, the i-th aggregated state feature is the first aggregated state feature, and the (i+1)-th aggregated state feature is the second aggregated state feature; in the graph data structure, the distance between the second neighbor node and the network node is greater than the distance between the first neighbor node and the network node.

[0254] The cross-layer feature aggregation module is used to obtain, for each network node, the resource status features corresponding to the first resource status data of the network node at a first time, based on the aggregated state features of the network node corresponding to each hidden layer.

[0255] Optionally, in the resource scheduling device, the scheduling module 803 is specifically used for:

[0256] Based on the target resource status characteristics of each network node, the resource dependencies between the network nodes are obtained, wherein the target resource status characteristics include the first resource status characteristics or the second resource status characteristics.

[0257] Based on the resource dependencies and the deep Q-network, a scheduling strategy is obtained;

[0258] Resource scheduling actions are performed according to the scheduling policy.

[0259] Optionally, the resource scheduling device further includes:

[0260] An evaluation module is used to evaluate the scheduling strategy based on at least one indicator and obtain a strategy evaluation result.

[0261] The reward module is used to obtain reward feedback from the deep Q network based on the evaluation results of the policy.

[0262] An adjustment module is used to adjust the model parameters of the deep Q-network based on the reward feedback.

[0263] It should be noted that the apparatus provided in this application embodiment can implement all the method steps implemented in the above resource scheduling method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0264] This application also provides a resource scheduling device, such as... Figure 9 As shown, it includes:

[0265] The processor 901, memory 902, transceiver 903, and a program or instructions stored in the memory 902 and executable on the processor 901; when the processor 901 executes the program or instructions, it implements the various processes of the above-described resource scheduling method embodiments and achieves the same technical effect. To avoid repetition, these will not be described again here.

[0266] The transceiver 903 is used to receive and send data under the control of the processor 901.

[0267] Among them, Figure 9 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically connecting various circuits of one or more processors represented by processor 901 and memory represented by memory 902. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. Transceiver 903 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. For different user equipment, the user interface 904 can also be an interface capable of connecting external or internal devices, including but not limited to keypads, displays, speakers, microphones, joysticks, etc.

[0268] The processor 901 is responsible for managing the bus architecture and general processing, while the memory 902 can store the data used by the processor 901 when performing operations.

[0269] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the resource scheduling method embodiments described above and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0270] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the above-described resource scheduling method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0271] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0272] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0273] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A resource scheduling method, characterized in that, include: Based on the first resource status data of each network node in the computing power network, a graph data structure is constructed. The graph data structure includes multiple nodes and edges connecting the nodes. Each node corresponds to a network node, and the node is used to indicate the first resource status data of the corresponding network node. Based on the graph convolutional neural network and the graph data structure, a first resource state feature corresponding to the first resource state data of each network node is obtained; Resource scheduling is performed based on the first resource state characteristics of each network node.

2. The method according to claim 1, characterized in that, After performing resource scheduling, the method further includes: Obtain the second resource status data for each of the network nodes; Based on the Long Short-Term Memory network and the second resource state data, the second resource state feature corresponding to the second resource state data of each network node is obtained; Based on the second resource status characteristics of each network node, determine whether the resource usage of the network node exceeds a preset threshold; If the resource usage of a network node exceeds a preset threshold, an early warning message is sent, and resource scheduling is performed based on the second resource status characteristics of each network node.

3. The method according to claim 1, characterized in that, The first resource status feature includes resource status features at a first time and resource status features at a second time, wherein the second time is the time after the first time. Based on the graph convolutional neural network and the graph data structure, a first resource state feature corresponding to the first resource state data of each network node is obtained, including: Based on the graph convolutional neural network and the graph data structure, the resource state features corresponding to the first time period of the first resource state data of each network node are obtained; Based on the Long Short-Term Memory network and the resource state characteristics at the first time, the resource state characteristics at the second time are obtained.

4. The method according to claim 3, characterized in that, The graph convolutional neural network includes convolutional layers, multiple hidden layers, and a cross-layer feature aggregation module, wherein... The convolutional layer is used to obtain the initial resource state features of the network node based on the first resource state data of each network node, and for each network node, to aggregate the initial resource state features of the first neighboring nodes of the network node into the initial resource state features of the network node to obtain the first aggregated state features of the network node. The i-th hidden layer is used to aggregate the initial resource state features of the (i+1)-th neighbor nodes of each network node into the i-th aggregated state feature to obtain the (i+1)-th aggregated state feature of the network node; where i is an integer greater than or equal to 1; when i=1, the (i+1)-th neighbor node is the second neighbor node, the i-th aggregated state feature is the first aggregated state feature, and the (i+1)-th aggregated state feature is the second aggregated state feature; in the graph data structure, the distance between the second neighbor node and the network node is greater than the distance between the first neighbor node and the network node. The cross-layer feature aggregation module is used to obtain, for each network node, the resource status features corresponding to the first resource status data of the network node at a first time, based on the aggregated state features of the network node corresponding to each hidden layer.

5. The method according to any one of claims 1 or 2, characterized in that, Performing resource scheduling includes: Based on the target resource status characteristics of each network node, the resource dependencies between the network nodes are obtained, wherein the target resource status characteristics include the first resource status characteristics or the second resource status characteristics. Based on the resource dependencies and the deep Q-network, a scheduling strategy is obtained; Resource scheduling actions are performed according to the scheduling policy.

6. The method according to claim 5, characterized in that, After performing resource scheduling, the method further includes: The scheduling strategy is evaluated based on at least one indicator to obtain a strategy evaluation result; Based on the evaluation results of the strategy, the reward feedback of the deep Q network is obtained; The model parameters of the deep Q-network are adjusted based on the reward feedback.

7. A resource scheduling device, characterized in that, include: The construction module is used to construct a graph data structure based on the first resource status data of each network node in the computing power network. The graph data structure includes multiple nodes and edges connecting the nodes. Each node corresponds to a network node, and the node is used to indicate the first resource status data of the corresponding network node. The acquisition module is used to obtain a first resource state feature corresponding to the first resource state data of each network node based on the graph convolutional neural network and the graph data structure. The scheduling module is used to perform resource scheduling based on the first resource status characteristics of each network node.

8. A resource scheduling device, characterized in that, include: A processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the processor, when executing the program or instructions, implements the resource scheduling method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the resource scheduling method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the resource scheduling method as described in any one of claims 1 to 6.

Images