Computing power resource scheduling method and system and storage medium

Abstract

The invention relates to the technical field of computing power scheduling, and discloses a computing power resource scheduling method and system and a storage medium. The computing power resource scheduling method comprises the steps of performing computing power node matching based on real-time computing power data of each computing power node and a computing task of to-be-allocated computing power resources to select a target computing power node; sequentially inputting the real-time computing power data and the historical computing power data of each target computing power node into a preset neural network model for computing power prediction to obtain predicted real-time computing power data, and determining a working mode of each computing power node; if the target computing power node is in the stable working mode, computing power resource scheduling is carried out according to historical computing power data of the target computing power node; and if the target computing power node is in the fluctuation working mode, computing power resource scheduling is carried out according to the real-time computing power data of the target computing power node and the predicted real-time computing power data. Intelligent computing power resource scheduling is achieved, computing power resource waste is avoided, computing power resource allocation tends to be more reasonable, and then the success rate of computing tasks is increased.

Description

Technical Field

[0001] The present invention relates to the field of computing power scheduling technology, and in particular to a computing power resource scheduling method, system and storage medium. Background Art

[0002] With the development of technologies such as big data, machine learning, and deep learning, technologies such as cloud computing and edge computing are also booming. Cloud computing platforms are generally built in a clustered manner, connecting multiple computing nodes together to form a large computing cluster. In a computing cluster, the computing power of each computing node may vary, so different computing tasks must be deployed to different computing nodes to ensure the computing efficiency of the tasks. In addition, due to the large number of computing nodes in the computing cluster and the constant changes in computing tasks, the computing resources of the computing nodes need to be reasonably configured and scheduled to fully utilize the computing resources, improve computing efficiency, and reduce computing energy consumption. Summary of the invention

[0003] The main purpose of the present invention is to provide a computing power resource scheduling method, system and storage medium, aiming to solve the technical problem that it is difficult to intelligently and reasonably schedule computing power resources when there are many nodes in the existing computing power cluster and diversified computing tasks, thereby reducing the success rate of computing tasks.

[0004] A first aspect of the present invention provides a computing resource scheduling method, the computing resource scheduling method comprising: Obtain the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; The target computing node is selected based on the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; Obtain the historical computing power data of each target computing power node, and input the real-time computing power data and historical computing power data of each target computing power node into the preset neural network model in turn for computing power prediction, and obtain the predicted real-time computing power data of each target computing power node; Based on the real-time computing power data and predicted real-time computing power data of each target computing power node, the working mode of each computing power node is determined respectively, and the working mode includes a stable working mode and a fluctuating working mode; If the target computing power node is in a stable working mode, computing power resources are scheduled according to the historical computing power data of the target computing power node; If the target computing power node is in the fluctuating working mode, computing power resources are scheduled according to the real-time computing power data and predicted real-time computing power data of the target computing power node.

[0005] A second aspect of the present invention provides a computing resource scheduling system, the computing resource scheduling system comprising: The acquisition module is used to obtain the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; A matching module is used to match computing nodes based on the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated, so as to select the target computing power node; The prediction module is used to obtain the historical computing power data of each target computing power node, and input the real-time computing power data and historical computing power data of each target computing power node into the preset neural network model in turn to perform computing power prediction, so as to obtain the predicted real-time computing power data of each target computing power node; A judgment module, used to judge the working mode of each computing power node based on the real-time computing power data and the predicted real-time computing power data of each target computing power node, wherein the working mode includes a stable working mode and a fluctuating working mode; The scheduling module is used to schedule computing power resources according to the historical computing power data of the target computing power node if the target computing power node is in a stable working mode; if the target computing power node is in a fluctuating working mode, the computing power resources are scheduled according to the real-time computing power data and predicted real-time computing power data of the target computing power node.

[0006] The third aspect of the present invention also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes the steps of any of the above-mentioned computing resource scheduling methods.

[0007] In the technical solution provided by the present invention, before computing power resource scheduling is performed, the computing power characteristics of the computing power node are first matched with the task characteristics of the computing task to be allocated computing power resources, so as to screen out the target computing power node that meets the requirements of the computing task, and ensure the adaptability of the computing power node and the task to be calculated; then the computing power of the computing power node is predicted by a deep learning neural network model to ensure that the predicted computing power data is closer to the actual computing power data, thereby improving the prediction accuracy and the accuracy of the computing power node resource scheduling, and then based on the real-time computing power data of each target computing power node and the predicted real-time computing power data, determine the working mode of each target computing power node and allocate resources based on different working modes. The present invention predicts the node computing power, grasps the fluctuation state of the node computing power in real time, allocates resources according to the node computing power, avoids the waste of computing power resources of the computing power node, makes the computing power resource allocation of the computing power node more rational, avoids the load imbalance caused by uneven resource allocation, and is not only conducive to improving the success rate of the computing task, but also ensures that the computing power node is in a fully utilized state, and further improves the node computing power resource utilization rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of an embodiment of a computing resource scheduling method in an embodiment of the present invention. DETAILED DESCRIPTION

[0009] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0010] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the computing resource scheduling method in the embodiment of the present invention includes: 101. Obtain the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; The scheduling system needs to first collect the real-time computing power data of all computing power nodes, including but not limited to CPU usage, memory occupancy, disk I / O speed, network bandwidth, etc. At the same time, the scheduling system also needs to obtain the computing tasks of the current computing power resources to be allocated, which usually include information such as the computing amount of the task, the type of resources required, and the computing priority.

[0011] Computing power node data collection can obtain computing power data in real time through monitoring software or API interfaces deployed on computing power nodes. The monitoring software or API interface should be able to send the node's computing power status information to the central scheduling system regularly (such as every second or every minute). Computing tasks are usually submitted to the scheduling system by users or applications. The scheduling system should provide a user-friendly interface or API that allows users to submit computing tasks, and include a detailed description of the task and the required resources.

[0012] 102. Match the computing nodes based on the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated, so as to select the target computing power node; In this embodiment, the system needs to analyze the specific computing power requirements of each computing task, including the required CPU, memory, disk and network resources, etc. At the same time, a suitable matching algorithm is used to compare the task requirements with the real-time computing power data of the node. For example, a greedy algorithm, a heuristic algorithm or a machine learning algorithm is used, and then one or more computing power nodes that are most suitable for executing the current task are selected as the target computing power nodes based on the matching results.

[0013] In an optional embodiment, the above step 102 further includes: 1021. Determine whether the computing power node matching condition is met based on the node running time in the real-time computing power data of each computing power node and the task execution time of the computing task of the computing power resource to be allocated; Collect the running time data of each computing power node in real time, which usually includes the node start running time, the expected end running time (or the current time, if the node is running continuously), etc. At the same time, obtain the task execution time of the computing task to be assigned from the task queue, including the task start time and the task end time. For each computing power node, check whether its running time interval (start time to end time) contains the task execution time interval (start time to end time) of the task to be assigned. If the running time interval of the node completely covers or partially covers the time interval of the task (that is, the task start time is not earlier than the node start time, and the task end time is not later than the node end time), the node is preliminarily considered to be available in time.

[0014] According to the above comparison results, all computing nodes that are available in time are screened out as candidate node sets for subsequent steps. If the candidate node set is empty, it means that no computing node can meet the time requirements of the task at present, and it is necessary to readjust the task execution time or wait for more nodes to become available. For nodes that run continuously, their end time can be regarded as a large value (such as a certain time point in the future), or simply not set the end time.

[0015] 1022. If the node running time of the computing power node is earlier than the task execution time of the computing task of the computing power resource to be allocated, determine whether the task execution time of the computing task of the computing power resource to be allocated is between the valley running time and the peak running time of the current computing power node; Get the trough and peak running times of the computing nodes from their real-time data. These times are usually determined based on the node's historical running data, load conditions, user behavior patterns, and other factors. The trough running time refers to the time period when the node load is low and the computing power utilization is low; the peak running time refers to the time period when the node load is high and the computing power utilization is high. For each candidate node selected in the time matching step, check whether the task execution time of the task to be assigned is between its trough running time and peak running time.

[0016] If the time interval of the task intersects with the trough to peak time interval of the node, the task execution time is considered to be within this interval. Based on the above comparison results, the computing nodes that meet the requirements in terms of time and load are further screened. If the task execution time of a node is not within its trough to peak time interval, it is removed from the candidate node set.

[0017] Considering that the trough and peak times of a node may change over time (such as daily or weekly periodic changes), the system needs to be able to dynamically update this time information. Machine learning algorithms or statistical methods can be used to predict future trough and peak times and adjust the matching strategy accordingly.

[0018] In actual implementation, determining the trough and peak times may be a complex process that requires consideration of multiple factors. For some special types of tasks (such as urgent tasks and high-priority tasks), it may be necessary to allocate computing resources during peak times.

[0019] 1023. If the task execution time of the computing task to be allocated computing resources is between the valley running time and the peak running time of the current computing node, the current computing node is determined as a candidate computing node; Based on the screening results of the first two steps, the final set of candidate nodes is determined. These nodes cover the task execution time in time, and the task execution time is between its trough running time and peak running time. If the candidate node set contains multiple nodes, and these nodes have different computing power characteristics (such as the number of CPUs, memory size, network bandwidth, etc.), the candidate nodes can be sorted according to these characteristics. The basis for sorting can be factors such as node computing power utilization, historical task completion, and user evaluation.

[0020] 1024. Perform feature matching based on the computing power characteristics of each candidate computing power node and the task characteristics of the computing task to be allocated computing power resources, and select the candidate computing power node with the highest feature matching degree as the target computing power node.

[0021] For each candidate node, extract its computing power features, such as CPU model, memory size, network bandwidth, storage capacity, computing power utilization, etc. For the computing tasks to be assigned, extract their task features, such as computing volume, memory requirements, IO requirements, network requirements, task type (such as CPU intensive, IO intensive, memory intensive, etc.), etc.

[0022] Design a feature matching algorithm to calculate the matching degree between the candidate node features and the task features. The algorithm can be implemented based on weighted summation, distance metrics (such as Euclidean distance, Manhattan distance), similarity metrics (such as cosine similarity, Jaccard similarity), etc. In the weighted summation method, each feature can be assigned a weight to indicate the importance of the feature in the matching. The weight can be adjusted according to actual needs. For each candidate node, use the feature matching algorithm to calculate its matching degree with the task. Sort the candidate nodes according to the matching degree, and select the node with the highest matching degree as the target computing power node. Consider using machine learning algorithms to optimize the feature matching process and improve matching accuracy and efficiency.

[0023] 103. Obtain historical computing power data of each target computing power node, and sequentially input the real-time computing power data and historical computing power data of each target computing power node into a preset neural network model for computing power prediction, to obtain predicted real-time computing power data of each target computing power node; In this embodiment, the computing power data of the target computing power node is first obtained from the historical database. The computing power data obtained should cover a sufficiently long period of time to reflect the computing power change trend of the node. At the same time, a suitable neural network model, such as a long short-term memory network or a gated recurrent unit, is selected, and the historical computing power data is input into the neural network model for training until the model achieves satisfactory prediction accuracy. During the training process, methods such as cross-validation can be used to prevent overfitting.

[0024] After the model training is completed, the real-time computing power data and historical computing power data of the target computing power node can be input into the trained neural network model to obtain the predicted future computing power data. The predicted data will be used for subsequent work mode judgment and resource scheduling decisions.

[0025] In an optional embodiment, the neural network model is obtained by the following training method: Obtain the historical real-time computing power data and historical predicted computing power data of each computing power node, where the historical computing power data of each period includes the error characteristic parameters of the actual computing power value and the predicted computing power value corresponding to the timestamp; Constructing an initial neural network model, wherein the input layer of the initial neural network model inputs historical real-time computing power data and historical predicted computing power data, and the output layer outputs the current predicted real-time computing power data predicted by the computing power node; According to the historical real-time computing power data and the historical predicted computing power data of each computing power node, a loss function of the initial neural network model is constructed, and the initial neural network model is trained to obtain a trained neural network model; Among them, the hidden layer input of the initial neural network model is as follows:

[0026] in, Represents the historical real-time computing power data vector of the kth input of the hidden layer computing power node i, Represents the historical predicted computing power data vector of the k-th input of the hidden layer computing power node i, is the historical real-time computing power data of the kth input of computing power node i, is the input weight of the historical real-time computing power data of the kth input of computing power node i, is the bias term of computing power node i, is the input weight coefficient of the kth input, N is the number of neurons in the input layer, f is the activation function, is the historical predicted computing power data of the kth input of computing power node i, is the input weight of the historical predicted computing power data of the kth input of computing power node i, is the bias term of computing power node i, is the input weight coefficient of the kth input, and M is the number of neurons in the input layer; Among them, the loss function of the initial neural network model is as follows:

[0027] in, represents the loss function of the initial neural network model, is the computing power data distribution matrix of computing power node i and computing power node j in the historical time period, S is the number of computing power node samples, is the real-time computing power data of computing power node j, Represents the predicted computing power data of computing power node j predicted by the model.

[0028] 104. Based on the real-time computing power data and the predicted real-time computing power data of each target computing power node, determine the working mode of each computing power node respectively, wherein the working mode includes a stable working mode and / or a fluctuating working mode; In this embodiment, the stable working mode means that the computing power data of the computing power node remains relatively stable over a period of time with a small fluctuation range; the fluctuating working mode means that the computing power data of the node changes frequently with a large fluctuation range. Specifically, statistical methods (such as variance, standard deviation, etc.) or machine learning algorithms (such as clustering algorithms) can be used to determine the working mode of the node, and finally the judgment results are stored in the database for subsequent resource scheduling decisions.

[0029] 105. If the target computing power node is in a stable working mode, computing power resources are scheduled according to the historical computing power data of the target computing power node; When the target computing power node is in stable working mode, it means that the computing power resources of the current target computing power node are relatively stable, and resources can be scheduled based on historical computing power data.

[0030] In this embodiment, in a stable working mode, a relatively simple resource allocation strategy can be adopted, such as resource allocation based on historical average computing power. Specifically, the average computing power of the node can be calculated based on its historical computing power data, and then resources can be allocated to those nodes whose average computing power meets the task requirements according to the task requirements. According to the resource allocation strategy, a scheduling instruction is sent to the target computing power node to instruct it to start executing the computing task. The scheduling instruction should contain detailed information about the task, the amount of resources required, and the execution time. In addition, during the execution process, the system should continuously monitor the computing power data of the node. If the computing power of the target computing power node is found to have changed significantly, its working mode needs to be re-evaluated and the resource allocation strategy needs to be adjusted.

[0031] 106. If the target computing power node is in the fluctuating working mode, computing power resources are scheduled according to the real-time computing power data and the predicted real-time computing power data of the target computing power node.

[0032] When the target computing power node is in the fluctuating working mode, it means that the computing power resources of the current target computing power node change frequently, and more flexible resource scheduling is required based on real-time computing power data and predicted computing power data.

[0033] In this embodiment, in the fluctuating working mode, a more dynamic resource allocation strategy needs to be adopted. This can be achieved by monitoring the computing power data of the node in real time and dynamically adjusting the resource allocation according to the prediction results. For example, a reinforcement learning algorithm can be used to optimize the resource allocation strategy so that the system can minimize resource waste while meeting task requirements. Due to the large fluctuations in the computing power of the node, resource allocation may need to be adjusted frequently. Therefore, the system should support elastic scheduling, that is, it should be able to dynamically increase or decrease the amount of resources allocated to the node based on the real-time computing power data and prediction results.

[0034] In addition, in this embodiment, the target computing power node is in the fluctuating working mode, and the computing power fluctuation may cause task execution failure or delay. Therefore, the system should have risk management and fault tolerance mechanisms, such as task retry, resource backup, etc., to ensure the smooth execution of tasks. At the same time, the system should send scheduling instructions to the target computing power node and execute computing tasks. The system should continuously monitor the computing power data and task execution of the node to timely discover and deal with potential problems.

[0035] In an optional embodiment, the above step 106 further includes: 1061. If the target computing power node is in the fluctuating working mode, the numerical difference between the real-time computing power data of the target computing power node and the predicted real-time computing power data is compared in real time; In this embodiment, once it is confirmed that the node is in the fluctuating working mode, the system needs to obtain the computing power data of the node in real time. This can be achieved through API calls, log analysis, or by collecting data directly from the node. Real-time computing power data may include indicators such as CPU usage, memory occupancy, and GPU computing power. At the same time, the system also needs to obtain the predicted real-time computing power data for the node, and then compare the numerical difference between the real-time computing power data and the predicted real-time computing power data in real time. This is done by calculating the absolute difference or relative difference between the two. The size of the difference reflects the degree of deviation between the actual computing power and the predicted computing power.

[0036] 1062. If the numerical difference exceeds the preset difference range, it is determined that the target computing power node currently has a computing power anomaly, and the number of anomalies of the target computing power node is increased by one; The system needs to set a preset difference range for each computing power indicator. The range can be determined based on the historical performance of the node, business needs, or industry standards. The difference range can be a fixed value or a percentage based on the predicted value. The system compares the numerical difference between the real-time computing power data and the predicted computing power data to check whether it exceeds the preset difference range. If it is found that the numerical difference exceeds the preset range, it is determined that the target computing power node currently has a computing power anomaly. The system needs to maintain a data structure (such as a database table or memory data structure) that records the number of anomalies and increases the number of anomalies of the target computing power node by one.

[0037] 1063. When the cumulative number of abnormalities of the target computing power node is within the first preset range, it is determined that the target computing power node is in a regular fluctuation state; when the cumulative number of abnormalities of the target computing power node is within the second preset range, it is determined that the target computing power node is in an irregular fluctuation mode; The system needs to set preset ranges for the cumulative number of exceptions for regular fluctuation states and irregular fluctuation modes. These ranges can be determined based on the historical performance of the node, business needs, or industry standards. For example, the first preset range may be 0-5 times, and the second preset range may be greater than 6 times. The system checks the cumulative number of exceptions of the target computing power node to determine whether it is within the preset range. This can be achieved by querying the abnormal number data structure maintained in step 1062. Based on the range of the cumulative number of exceptions, the system determines whether the target computing power node is in a regular fluctuation state or an irregular fluctuation mode.

[0038] 1064. If the target computing power node is in a regular fluctuation state, the computing power fluctuation threshold of the target computing power node is determined based on the historical computing power data of the target computing power node, and computing power resources are scheduled according to the computing power fluctuation threshold; The system needs to collect historical computing power data of the target computing power node, including real-time computing power data, predicted computing power data and the differences between them in the past period of time. Based on the historical computing power data, the system uses statistical analysis methods (such as mean, standard deviation, percentile, etc.) to determine the computing power fluctuation threshold. This threshold reflects the normal fluctuation range of the computing power of the node under a regular fluctuation state. Once the computing power fluctuation threshold is determined, the system can dynamically adjust the computing power resources according to this threshold. For example, when the real-time computing power data approaches or exceeds the threshold, the system can trigger resource expansion operations (such as increasing the number of CPU cores, allocating more memory, etc.) to ensure that the computing power requirements of the node are met.

[0039] 1065. If the target computing power node is in an irregular fluctuation state, the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node are obtained; The system needs to obtain the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node in real time. This can be achieved through API calls, log analysis, or by directly collecting data from the node. Similar to step 1061, this is a specific acquisition operation for irregular fluctuation states. After acquiring the data, the system should perform necessary data verification to ensure the accuracy and completeness of the data. For example, you can check whether the data is missing or exceeds a reasonable range.

[0040] 1066. Calculate the difference between the latest real-time computing power data and the latest predicted real-time computing power data to obtain the computing power fluctuation range of the target computing power node; First, pre-process the latest real-time computing power data and the latest predicted real-time computing power data to ensure that the data format is consistent and the timestamps are aligned. Then, the difference between the real-time computing power data and the predicted computing power data at each time point is calculated to obtain the computing power fluctuation difference sequence. The difference calculation formula is: computing power fluctuation difference = real-time computing power data - predicted computing power data.

[0041] Perform statistical analysis on the hashrate fluctuation difference sequence and calculate the average, maximum, minimum and other statistics of its absolute value. Use these statistics as the measurement indicators of the hashrate fluctuation amplitude. Of course, more complex measurement methods can also be used, such as calculating the variance and standard deviation of the fluctuation difference, to more comprehensively reflect the amplitude and stability of the hashrate fluctuation. The calculated hashrate fluctuation amplitude is stored in the database for query and analysis in subsequent steps.

[0042] 1067. Based on the computing power fluctuation amplitude of the target computing power node, determine whether the target computing power node has a computing power shortage and determine the computing power prediction accuracy of the target computing power node in the fluctuating working mode; A threshold for computing power shortage is set in advance. The threshold can be set according to the actual needs of the system. For example, the real-time computing power data can be lower than a preset value, or the difference between the real-time computing power data and the predicted computing power data exceeds a certain range. If the real-time computing power data of the target computing power node is lower than the computing power shortage threshold, or the computing power fluctuation range exceeds the set fluctuation range, the node is considered to have computing power shortage.

[0043] For target computing power nodes in fluctuating working mode, it is necessary to evaluate the accuracy of their computing power prediction. A variety of evaluation indicators can be used to measure the prediction accuracy, such as mean square error, mean absolute error, root mean square error, etc. When calculating these evaluation indicators, it is necessary to compare the predicted computing power data with the actual computing power data. According to the value of the evaluation indicator, the prediction accuracy can be divided into different levels such as high, medium, and low. The results of the judgment of computing power shortage and the judgment of prediction accuracy are recorded to provide a basis for the decision-making of subsequent steps.

[0044] In an optional embodiment, the above step 1067 further includes: S101. If the value of the computing power fluctuation amplitude of the target computing power node is a negative number, it is determined that the target computing power node has a computing power shortage. If the value of the computing power fluctuation amplitude of the target computing power node is a positive number, it is determined that the target computing power node has no computing power shortage. Computing power fluctuation range = (current computing power - expected computing power) / expected computing power * 100%, where the current computing power is the actual computing power of the target computing power node at a certain moment, and the expected computing power is the computing power that the target computing power node should achieve at that moment.

[0045] Obtain the current and expected computing power of the target computing power node through the monitoring or log system, and calculate the computing power fluctuation range. If the computing power fluctuation range is <0, it is determined that the target computing power node has a computing power shortage. If the computing power fluctuation range is >= 0, it is determined that the target computing power node has no computing power shortage.

[0046] S102, obtaining the maximum historical computing power fluctuation range among the historical computing power fluctuation ranges of the target computing power node in the fluctuation working mode; S103. When the difference between the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node exceeds the deviation threshold range, the difference between the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node is calculated to obtain the real-time computing power fluctuation range of the target computing power node; In this embodiment, a deviation threshold is preset to determine whether the difference between the latest real-time computing power data and the latest predicted real-time computing power data exceeds an acceptable range.

[0047] Obtain the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node through the monitoring or log system. Calculate the difference between the latest real-time computing power data and the latest predicted real-time computing power data. If the difference exceeds the deviation threshold, proceed to the next step to calculate the fluctuation range of the real-time computing power; otherwise, do not perform this step.

[0048] Real-time computing power fluctuation range = (latest real-time computing power data - latest predicted real-time computing power data) / latest predicted real-time computing power data * 100% S104. Calculate the ratio of the real-time computing power fluctuation amplitude of the target computing power node to the maximum historical computing power fluctuation amplitude. If the ratio is greater than or equal to a preset ratio threshold, determine that the computing power prediction accuracy of the target computing power node is at a high level. If the ratio is less than the ratio threshold, determine that the computing power prediction accuracy of the target computing power node is at a low level.

[0049] Set a preset ratio threshold to determine the accuracy level of the computing power forecast. Obtain the real-time computing power fluctuation range of the target computing power node through the previous steps. Obtain the maximum historical computing power fluctuation range of the target computing power node through the previous steps.

[0050] Ratio = real-time computing power fluctuation range / maximum historical computing power fluctuation range Determine the level of computing power prediction accuracy: If the ratio >= the preset ratio threshold, it is judged as a high level.

[0051] If the ratio is less than the preset ratio threshold, it is judged as a low level.

[0052] 1068. If the target computing power node has a computing power shortage and the computing power prediction accuracy is high, the type of the target computing power node is marked as a stable computing power shortage node; 1069. If the target computing power node has a computing power shortage and the computing power prediction accuracy is low, the type of the target computing power node is marked as a sudden computing power shortage node; Based on the results of the previous steps, determine whether the target computing power node meets both the conditions of "generating computing power shortage" and "high computing power prediction accuracy". If both conditions are met, proceed to the next step.

[0053] If the type of the target computing power node is recorded as "stable lack of computing power node", although the node has a lack of computing power, its computing power fluctuation has a certain regularity, and the prediction model can better predict its future computing power change trend. For stable lack of computing power nodes, targeted computing power resource scheduling strategies can be formulated. For example, a certain amount of computing power resources can be reserved in advance, or when computing power is missing, resources can be scheduled from other nodes to make up for it.

[0054] If the type of the target computing power node is "sudden lack of computing power node", the computing power loss of the node is sudden, and the prediction model cannot accurately predict its future computing power change trend. For nodes with sudden lack of computing power, a more flexible computing power resource scheduling strategy needs to be formulated. For example, the computing power status of the node can be monitored in real time. Once the computing power loss is detected, resources can be immediately dispatched from other nodes to deal with it. At the same time, the monitoring and early warning of these nodes can be strengthened to timely discover and deal with potential computing power problems.

[0055] 10610. Perform computing resource scheduling based on the type of target computing node.

[0056] Select the corresponding computing resource scheduling strategy according to the type of target computing power node (stable computing power loss node or sudden computing power loss node). For stable computing power loss nodes, you can use strategies such as reserved resources and priority scheduling. For sudden computing power loss nodes, you need to use a more flexible and faster resource scheduling strategy.

[0057] In this embodiment, before resource scheduling is performed, the available computing resources in the system may be further evaluated, including calculating the remaining computing power of each node, evaluating factors such as communication delay and bandwidth between different nodes, etc. Then, based on the results of resource evaluation, a specific computing resource scheduling plan is formulated, including determining which nodes to schedule resources from, how many resources to schedule, when to schedule, etc. The scheduling decision is converted into a specific execution operation, such as starting or stopping certain computing tasks, adjusting the load distribution of nodes, establishing or disconnecting communication connections between nodes, etc.

[0058] 1. The type of the target computing power node is a stable missing computing power node. In this embodiment, when the type of the target computing power node is a stable missing computing power node, the computing power resource scheduling is performed in the following manner: 1.1. Obtain the real-time computing power data and predicted real-time computing power data of the stable missing computing power node at different times and perform regression operations to obtain the stable predicted computing power of the stable missing computing power node within the deviation threshold range; In this embodiment, since the computing power data may fluctuate greatly at different time points, in order to improve the stability and accuracy of the prediction model, it is necessary to normalize the real-time computing power data and the predicted real-time computing power data obtained at different times, so as to convert the data to the same scale. Among them, the predicted real-time computing power data needs to be predicted by the prediction model.

[0059] For example, historical real-time computing power data is used in advance to train the selected prediction algorithm (such as a recurrent neural network), and the model is optimized through methods such as cross-validation to improve the accuracy and robustness of the prediction. During the training process, the parameters of the model, such as the learning rate and the number of iterations, can be adjusted to achieve the best prediction effect. After the model training is completed, the real-time computing power data is used as input to obtain the predicted real-time computing power data through the prediction model. Then, the predicted real-time computing power data is regressed with the real-time computing power data to evaluate the accuracy of the prediction.

[0060] Based on the regression operation, a deviation threshold (such as ±10%) is set, and the deviation between the predicted real-time computing power data and the real-time computing power data within the threshold range is regarded as the stable predicted computing power. This can be achieved by calculating the error between the predicted value and the actual value, and filtering out the data points whose error is within the threshold range.

[0061] 1.2. Add the stable predicted computing power and the predicted real-time computing power of the stable missing computing power node to obtain the predicted real-time computing power increase value of the stable missing computing power node, and subtract the stable predicted computing power and the predicted real-time computing power of the stable missing computing power node to obtain the predicted real-time computing power decrease value of the stable missing computing power node; After obtaining the stable predicted computing power, add it to the predicted real-time computing power data at the same time point. This will get a value indicating the trend of computing power increase, that is, the predicted real-time computing power increase value. This value reflects the part of the node computing power that may increase in the future. Similarly, subtract the stable predicted computing power from the predicted real-time computing power data to get a value indicating the trend of computing power decrease, that is, the predicted real-time computing power decrease value. This value reflects the part of the node computing power that may decrease in the future.

[0062] 1.3. The computing power resources corresponding to the predicted real-time computing power increase value of the stable missing computing power node are dispatched to the stable missing computing power node, and the computing power resources corresponding to the predicted real-time computing power decrease value of the stable missing computing power node are dispatched to other computing power nodes.

[0063] According to the size and direction of the predicted real-time computing power increase and decrease, formulate computing power resource scheduling strategies, including determining the allocation ratio of computing power resources, scheduling timing, priority, etc. When formulating strategies, it is necessary to comprehensively consider factors such as the node's computing power requirements, resource utilization, and task priority.

[0064] According to the formulated computing power resource scheduling strategy, the computing power resources corresponding to the predicted real-time computing power increase value are allocated to the stable computing power missing nodes. This can be achieved by adjusting the computing power configuration of the node, optimizing task allocation, etc. At the same time, it is also necessary to pay attention to the real-time computing power changes of the node so as to adjust the resource allocation plan in time. For the computing power resources corresponding to the predicted real-time computing power reduction value, they can be recovered and reallocated to other computing power nodes. This can be achieved by releasing idle resources, optimizing task scheduling, etc. In the process of resource recovery and redistribution, it is necessary to ensure the effective use of resources and avoid waste.

[0065] (2) The type of the target computing power node is a sudden missing computing power node In this embodiment, when the type of the target computing power node is a sudden missing computing power node, the computing power resource scheduling is performed in the following manner: 2.1. Classify the nodes with sudden missing computing power into first missing computing power nodes whose computing power missing threshold is lower than the preset computing power threshold and second missing computing power nodes whose computing power missing threshold is greater than or equal to the computing power threshold; The system needs to define a preset computing power threshold. This threshold can be set based on historical data, business needs, or system load. For example, if the system average computing power is 100 units, a threshold of 80 units can be set, and nodes below this value will be considered to have insufficient computing power. For each node, calculate its current computing power loss threshold. This is calculated by comparing the current computing power of the node with its maximum computing power or expected computing power. For example, if the maximum computing power of the node is 120 units and the current computing power is 60 units, then the computing power loss threshold is 60 units (i.e. half of the maximum computing power).

[0066] Based on the preset computing power threshold, nodes are divided into two categories: The first missing computing power node: a node whose computing power missing threshold is lower than the preset computing power threshold.

[0067] Second missing computing power node: a node whose computing power missing threshold is greater than or equal to the preset computing power threshold.

[0068] 2.2. If the type of the suddenly missing computing power node is the first missing computing power node, obtain the adjacent computing power nodes of the first missing computing power node, predict the first computing power threshold of the first missing computing power node according to the real-time computing power data of the adjacent computing power nodes, and dispatch the computing power resources in the first missing computing power node to the adjacent computing power nodes and the first missing computing power node according to the preset ratio; For each first missing computing power node, its adjacent computing power node needs to be found. This can be achieved through a network topology diagram or the connection relationship between nodes. For example, the connection relationship between nodes can be represented by an adjacency list or adjacency matrix of a graph. A machine learning algorithm or statistical model is used to predict the first computing power threshold of the first missing computing power node. This can be achieved based on factors such as the real-time computing power data of adjacent nodes, historical computing power data, and correlation between nodes. For example, algorithms such as linear regression, decision trees, or neural networks can be used to establish a prediction model.

[0069] According to the predicted first computing power threshold and the preset ratio, the computing power resources in the first missing computing power node are dispatched to the adjacent computing power nodes and the first missing computing power node. This can be achieved by adjusting the computing power allocation strategy or task scheduling strategy of the node. For example, a part of the computing power resources can be migrated from the first missing computing power node to the adjacent node to balance the computing power load of the entire system. Finally, the computing power status of the first missing computing power node and the adjacent node is updated and recorded in the system log or database.

[0070] 2.3. If the type of the suddenly missing computing power node is the second missing computing power node, the computing power data changes of the second missing computing power node within the preset time are monitored in real time; Within the preset time, the system needs to continuously collect the real-time computing power data of the second missing computing power node, including the node's CPU usage, memory occupancy, disk I / O speed, network bandwidth and other key indicators. Perform change analysis on the collected computing power data to identify the computing power change trend of the node. For example, this can be achieved through time series analysis algorithms to capture the periodicity, trend and random components in the data.

[0071] 2.4. If the real-time computing power data of the second missing computing power node exceeds the first threshold range or the second threshold range, the second missing computing power node is determined to be a node with a sudden change in computing power; if the real-time computing power data of the second missing computing power node is within the first threshold range or the second threshold range, the second missing computing power node is determined to be a node with a continuously reduced computing power; The first threshold range and the second threshold range are set according to the system's tolerance for computing power fluctuations. The first threshold range is usually narrower and is used to capture significant computing power mutations; the second threshold range is wider and is used to identify the trend of gradually decreasing computing power. These thresholds can be determined based on historical data, business needs, and system stability requirements. During real-time monitoring, the system needs to continuously compare the real-time computing power data of the second missing computing power node with the set threshold range. Based on the comparison results of the real-time computing power data and the threshold range, the system divides the second missing computing power nodes into two categories: Sudden change computing power nodes: nodes whose real-time computing power data exceeds the first threshold range or the second threshold range. The computing power of such nodes changes significantly and unpredictably, which may be caused by hardware failure, software anomalies, or external attacks.

[0072] Nodes with continuously decreasing computing power: nodes whose real-time computing power data is within the first threshold range or the second threshold range. The computing power of such nodes is gradually decreasing, which may be caused by resource overload, hardware aging, or software performance bottlenecks.

[0073] 2.5. If the second missing computing power node is a node with sudden change in computing power, the second missing computing power node will be eliminated. If the second missing computing power node is a node with continuously reduced computing power, the computing power resources in the second missing computing power node will be dispatched to other computing power nodes according to the preset ratio.

[0074] For the second missing computing power node that is determined to be a node with sudden change in computing power, the system needs to immediately remove it from the computing power resource pool. This can be achieved by stopping the node's service, releasing the resources it occupies, and updating the node status. The purpose of removal is to prevent the node from further affecting the overall performance of the system. For the second missing computing power node that is determined to be a node with continuously reduced computing power, the system needs to dispatch its remaining computing power resources to other computing power nodes according to the preset ratio. The scheduling ratio can be determined based on the node's real-time computing power requirements, priority, and the overall computing power allocation strategy of the system. The purpose of scheduling is to maximize the use of system resources and ensure business continuity and stability. After completing the removal or scheduling operation, the system needs to update the node's status and resource configuration information, including marking the status of the removed node as "unavailable", updating the scheduled node computing power resources to the resource management system, and adjusting the load balancing strategy of the relevant nodes.

[0075] In one embodiment, in order to further improve the efficiency and accuracy of computing resource scheduling, this embodiment further provides a computing node recommendation function. In this embodiment, the computing resource scheduling method further includes: S201. Obtain the average resource utilization rate and the maximum resource utilization rate of each target computing power node corresponding to each time period in the historical cycle; Get the relevant data of each target computing power node, including timestamp, computing power node ID, resource utilization (CPU, memory, disk, network, etc.), etc. Then calculate the average resource utilization and the maximum resource utilization. The historical period can be divided into multiple time periods, each of which can be hours, days, weeks, etc., depending on business needs. For each time period, calculate the average resource utilization of each target computing power node. This usually involves summing the resource utilization of all sampling points in the time period and then dividing it by the number of sampling points. Similarly, for each time period, find the maximum resource utilization of each target computing power node. Finally, store the calculated average resource utilization and maximum resource utilization in the database for use in subsequent steps.

[0076] S202, subtracting the maximum resource utilization rate of each target computing power node corresponding to each time period in the historical cycle from the average resource utilization rate to obtain the resource utilization rate difference of each target computing power node corresponding to each time period in the historical cycle; The average resource utilization rate and the maximum resource utilization rate calculated in step S201 are read from the database. For each target computing power node, in each time period, the maximum resource utilization rate is subtracted from the average resource utilization rate to obtain the resource utilization rate difference. This difference reflects the fluctuation of resource utilization rate, that is, the difference between the peak value and the average value of resource utilization rate.

[0077] S203, multiplying the computing power resources corresponding to each time period of each target computing power node in the historical cycle by the difference in resource utilization rate corresponding to the corresponding time period, to obtain the computing power resource utilization rate corresponding to each time period of each target computing power node in the historical cycle; The resource utilization difference calculated in step S202 and the computing power resources (such as the number of CPU cores, memory capacity, etc.) of the target computing power node in each time period are read from the database. For each target computing power node, in each time period, the computing power resources are multiplied by the corresponding resource utilization difference to obtain the computing power resource utilization rate. This value reflects the actual usage of computing power resources when the resource utilization rate fluctuates.

[0078] S204, comparing the computing resource utilization rate of each target computing node in each period of the historical cycle with the set resource utilization rate. If the computing resource utilization rate of each target computing node in each period of the historical cycle is higher than or equal to the set resource utilization rate, the corresponding target computing node is used as the historical recommended computing node for the next computing resource scheduling; According to business needs, a resource utilization threshold is set. This threshold can be fixed or dynamically adjusted, depending on business needs and resource management strategies. The computing power resource utilization calculated in step S203 is read from the database. For each target computing power node, the computing power resource utilization is compared with the set resource utilization threshold in each period of the historical cycle. If the computing power resource utilization in all periods is higher than or equal to the threshold, the target computing power node is used as the historical recommended computing power node for the next computing power resource scheduling.

[0079] S205, obtaining actual scheduling resources and estimated scheduling resources of each historically recommended computing power node, wherein the estimated scheduling resources are predicted by a preset intelligent prediction model; Read the actual scheduling resource data of the historically recommended computing nodes in the past period from the database, including the allocation and usage of computing resources. According to business needs, extract the features that affect the scheduling of computing resources, such as time, geographical location of computing nodes, hardware configuration, historical resource usage, etc. According to the characteristics of the data and business needs, select a suitable intelligent prediction model, such as a time series analysis model, a machine learning model (such as a regression model, decision tree, random forest, etc.) or a deep learning model (such as a neural network). Use the preprocessed data to train the intelligent prediction model to obtain the model parameters. Adjust the model parameters or select another model based on the evaluation results. Use the trained intelligent prediction model to predict the expected scheduling resources of each historically recommended computing node in the future period based on the current feature data.

[0080] S206: Determine the initial value of computing resource scheduling corresponding to each historically recommended computing node based on the actual scheduling resources and the estimated scheduling resources of each historically recommended computing node; The actual scheduled resources and the expected scheduled resources obtained in step S205 are read from the database. For each historically recommended computing power node, the initial value of computing power resource scheduling is calculated based on the actual demand and business strategy, combined with the actual scheduled resources and the expected scheduled resources. This initial value can be the average value, weighted average value or other statistics of the actual scheduled resources and the expected scheduled resources, depending on the business demand and resource management strategy.

[0081] S207, compare the resource scheduling initial value corresponding to each historically recommended computing power node with the set resource scheduling initial value. If the resource scheduling initial value corresponding to each historically recommended computing power node is higher than or equal to the set resource scheduling initial value, then use the corresponding historically recommended node as the historically recommended computing power node for the required resource scheduling final value; if the resource scheduling initial value corresponding to each historically recommended computing power node is lower than the set resource scheduling initial value, then use the corresponding historically recommended computing power node as the historically recommended computing power node for limiting the resource scheduling final value; According to business needs, set a resource scheduling initial value threshold. This threshold can be fixed or dynamically adjusted, depending on business needs and resource management strategies. Read the computing power resource scheduling initial value calculated in step S206 from the database. For each historically recommended computing power node, compare its computing power resource scheduling initial value with the set resource scheduling initial value threshold. If the computing power resource scheduling initial value is higher than or equal to the threshold, the historically recommended computing power node is used as the historically recommended computing power node for the required resource scheduling final value; if the computing power resource scheduling initial value is lower than the threshold, it is used as the historically recommended computing power node for the restricted resource scheduling final value.

[0082] S208, inputting the final scheduling value of each demand resource and the final scheduling value of the restricted resource into the intelligent prediction model for prediction, and obtaining the schedulable resources corresponding to each historically recommended computing power node; According to business needs, extract features that affect computing resource scheduling, such as time, geographic location of computing nodes, hardware configuration, historical resource usage, etc. For historically recommended computing nodes with demand resource scheduling final values ​​and restricted resource scheduling final values, extract corresponding feature data respectively. Use the intelligent prediction model trained in step S205 to predict schedulable resources, specifically: input the feature data of historically recommended computing nodes with demand resource scheduling final values ​​and restricted resource scheduling final values ​​into the intelligent prediction model respectively, and predict the schedulable resources of each historically recommended computing node in the future period of time.

[0083] S209: perform difference calculation between the schedulable resources corresponding to each historically recommended computing power node and the initial resource scheduling value corresponding to the corresponding historically recommended computing power node to obtain the final resource scheduling value of the corresponding historically recommended computing power node.

[0084] The schedulable resources corresponding to each historically recommended computing power node predicted in step S208 are read from the database. At the same time, the initial resource scheduling values ​​corresponding to each historically recommended computing power node calculated in step S206 are read from the database. Ensure that the schedulable resources of each historically recommended computing power node and its corresponding initial resource scheduling values ​​can be correctly matched. This is usually achieved through a unique identifier (such as an ID) of the computing power node. For each historically recommended computing power node, the difference between its schedulable resources and its initial resource scheduling value is calculated. The difference reflects the adjustment amount from the initial value to the final value, that is, the adjustment that needs to be made to the initial value based on the prediction results of the intelligent prediction model.

[0085] The difference calculation formula can be expressed as: resource scheduling final value = schedulable resources - resource scheduling initial value. The difference is directly used as the adjustment amount, so the resource scheduling final value is actually equal to the initial value plus the difference. It should be noted that "difference calculation" does not mean that the result of direct subtraction is the final scheduling value, but uses this difference to indicate the adjustment direction and amplitude from the initial value to the final value. The final scheduling value (i.e. the resource scheduling final value) should be the initial value plus (or minus, depending on the positive or negative difference) the adjustment amount.

[0086] In addition, if the calculated difference is very large (positive or negative), you may need to further check the accuracy of the data and the predictive performance of the model. If the difference exceeds the preset reasonable range, you can consider using other strategies for adjustment, such as limiting the maximum adjustment range, using smoothing algorithms, etc.

[0087] The above describes the computing power resource scheduling method in the embodiment of the present invention. The following describes the computing power resource scheduling system in the embodiment of the present invention. An embodiment of the computing power resource scheduling system in the embodiment of the present invention includes: The acquisition module is used to obtain the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; A matching module is used to match computing nodes based on the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated, so as to select the target computing power node; The prediction module is used to obtain the historical computing power data of each target computing power node, and input the real-time computing power data and historical computing power data of each target computing power node into the preset neural network model in turn to perform computing power prediction, so as to obtain the predicted real-time computing power data of each target computing power node; A judgment module, used to judge the working mode of each computing power node based on the real-time computing power data and the predicted real-time computing power data of each target computing power node, wherein the working mode includes a stable working mode and a fluctuating working mode; The scheduling module is used to schedule computing power resources according to the historical computing power data of the target computing power node if the target computing power node is in a stable working mode; if the target computing power node is in a fluctuating working mode, the computing power resources are scheduled according to the real-time computing power data and predicted real-time computing power data of the target computing power node.

[0088] Optionally, in one embodiment, the matching module is specifically used for: According to the node running time in the real-time computing power data of each computing power node and the task execution time of the computing task to be allocated computing power resources, it is judged whether the computing power node matching conditions are met; If the node running time of the computing power node is earlier than the task execution time of the computing task of the computing power resource to be allocated, then determine whether the task execution time of the computing task of the computing power resource to be allocated is between the valley running time and the peak running time of the current computing power node; If the task execution time of the computing task to be allocated computing resources is between the valley running time and the peak running time of the current computing node, the current computing node is determined as a candidate computing node; Feature matching is performed based on the computing power characteristics of each candidate computing power node and the task characteristics of the computing task to be allocated computing power resources, and the candidate computing power node with the highest feature matching degree is selected as the target computing power node.

[0089] Optionally, in one embodiment, the scheduling module is specifically used to: If the target computing power node is in the fluctuating working mode, the numerical difference between the real-time computing power data of the target computing power node and the predicted real-time computing power data is compared in real time; If the numerical difference exceeds the preset difference range, it is determined that the target computing power node currently has a computing power anomaly, and the number of anomalies of the target computing power node is increased by one; When the number of abnormalities accumulated by the target computing power node is within the first preset range, it is determined that the target computing power node is in a regular fluctuation state; when the number of abnormalities accumulated by the target computing power node is within the second preset range, it is determined that the target computing power node is in an irregular fluctuation mode; If the target computing power node is in a state of regular fluctuation, the computing power fluctuation threshold of the target computing power node is determined based on the historical computing power data of the target computing power node, and computing power resources are scheduled according to the computing power fluctuation threshold; If the target computing power node is in an irregular fluctuation state, obtain the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node; Calculate the difference between the latest real-time computing power data and the latest predicted real-time computing power data to obtain the computing power fluctuation range of the target computing power node; Based on the computing power fluctuation amplitude of the target computing power node, determine whether the target computing power node has computing power shortage and determine the computing power prediction accuracy of the target computing power node in the fluctuating working mode; If the target computing power node has a computing power shortage and the computing power prediction accuracy is high, the type of the target computing power node is marked as a stable computing power shortage node; If the target computing power node has a computing power shortage and the computing power prediction accuracy is low, the type of the target computing power node is marked as a sudden computing power shortage node; Computing resources are scheduled based on the type of target computing node.

[0090] Optionally, in one embodiment, the judgment module is specifically used to: If the value of the target computing power node's computing power fluctuation range is a negative number, it is determined that the target computing power node has a computing power shortage. If the value of the target computing power node's computing power fluctuation range is a positive number, it is determined that the target computing power node has no computing power shortage. Obtain the maximum historical computing power fluctuation range among the historical computing power fluctuation ranges of the target computing power node in the fluctuation working mode; When the difference between the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node exceeds the deviation threshold range, the difference between the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node is calculated to obtain the real-time computing power fluctuation range of the target computing power node; Calculate the ratio of the real-time computing power fluctuation amplitude of the target computing power node to the maximum historical computing power fluctuation amplitude. If the ratio is greater than or equal to the preset ratio threshold, the computing power prediction accuracy of the target computing power node is determined to be high. If the ratio is less than the ratio threshold, the computing power prediction accuracy of the target computing power node is determined to be low.

[0091] Optionally, in one embodiment, the scheduling module is further configured to: If the type of the target computing power node is a stable missing computing power node, the real-time computing power data and the predicted real-time computing power data of the stable missing computing power node at different times are obtained and regression operations are performed to obtain the stable predicted computing power of the stable missing computing power node within the deviation threshold range; Add the stable predicted computing power and the predicted real-time computing power of the stable missing computing power node to obtain the predicted real-time computing power increase value of the stable missing computing power node, and subtract the stable predicted computing power and the predicted real-time computing power of the stable missing computing power node to obtain the predicted real-time computing power decrease value of the stable missing computing power node; The computing power resources corresponding to the predicted real-time computing power increase value of the stable missing computing power node are dispatched to the stable missing computing power node, and the computing power resources corresponding to the predicted real-time computing power decrease value of the stable missing computing power node are dispatched to other computing power nodes.

[0092] Optionally, in one embodiment, the scheduling module is further configured to: If the type of the target computing power node is a sudden missing computing power node, the sudden missing computing power node is classified into a first missing computing power node whose computing power missing threshold is lower than a preset computing power threshold and a second missing computing power node whose computing power missing threshold is greater than or equal to the computing power threshold; If the type of the sudden missing computing power node is the first missing computing power node, the adjacent computing power nodes of the first missing computing power node are obtained, the first computing power threshold of the first missing computing power node is predicted according to the real-time computing power data of the adjacent computing power nodes, and the computing power resources in the first missing computing power node are dispatched to the adjacent computing power nodes and the first missing computing power node according to the preset ratio; If the type of the suddenly missing computing power node is the second missing computing power node, the computing power data change of the second missing computing power node within a preset time is monitored in real time; If the real-time computing power data of the second missing computing power node exceeds the first threshold range or the second threshold range, the second missing computing power node is determined to be a sudden change computing power node; if the real-time computing power data of the second missing computing power node is within the first threshold range or the second threshold range, the second missing computing power node is determined to be a continuously reduced computing power node; If the second missing computing power node is a node with a sudden change in computing power, the second missing computing power node will be eliminated. If the second missing computing power node is a node with a continuously reduced computing power, the computing power resources in the second missing computing power node will be dispatched to other computing power nodes according to a preset ratio.

[0093] Optionally, in one embodiment, the neural network model is obtained by the following training method: Obtain the historical real-time computing power data and historical predicted computing power data of each computing power node, where the historical computing power data of each period includes the error characteristic parameters of the actual computing power value and the predicted computing power value corresponding to the timestamp; Constructing an initial neural network model, wherein the input layer of the initial neural network model inputs historical real-time computing power data and historical predicted computing power data, and the output layer outputs the current predicted real-time computing power data predicted by the computing power node; According to the historical real-time computing power data and the historical predicted computing power data of each computing power node, a loss function of the initial neural network model is constructed, and the initial neural network model is trained to obtain a trained neural network model; Among them, the hidden layer input of the initial neural network model is as follows:

[0094] in, Represents the historical real-time computing power data vector of the kth input of the hidden layer computing power node i, Represents the historical predicted computing power data vector of the k-th input of the hidden layer computing power node i, is the historical real-time computing power data of the kth input of computing power node i, is the input weight of the historical real-time computing power data of the kth input of computing power node i, is the bias term of computing power node i, is the input weight coefficient of the kth input, N is the number of neurons in the input layer, f is the activation function, is the historical predicted computing power data of the kth input of computing power node i, is the input weight of the historical predicted computing power data of the kth input of computing power node i, is the bias term of computing power node i, is the input weight coefficient of the kth input, and M is the number of neurons in the input layer; Among them, the loss function of the initial neural network model is as follows:

[0095] in, represents the loss function of the initial neural network model, is the computing power data distribution matrix of computing power node i and computing power node j in the historical time period, S is the number of computing power node samples, is the real-time computing power data of computing power node j, Represents the predicted computing power data of computing power node j predicted by the model.

[0096] Optionally, in one embodiment, the computing resource scheduling device further includes: The utilization rate calculation module is used to obtain the average resource utilization rate and maximum resource utilization rate of each target computing power node in each period of the historical cycle; the maximum resource utilization rate of each target computing power node in each period of the historical cycle is subtracted from the average resource utilization rate to obtain the resource utilization rate difference of each target computing power node in each period of the historical cycle; the computing power resources corresponding to each period of the historical cycle of each target computing power node are multiplied by the resource utilization rate difference of the corresponding period to obtain the computing power resource utilization rate of each target computing power node in each period of the historical cycle; The utilization rate comparison module is used to compare the computing resource utilization rate of each target computing power node in each period of the historical cycle with the set resource utilization rate. If the computing resource utilization rate of each target computing power node in each period of the historical cycle is higher than or equal to the set resource utilization rate, the corresponding target computing power node will be used as the historical recommended computing power node for the next computing power resource scheduling; The node recommendation module is used to obtain the actual scheduling resources and expected scheduling resources of each historically recommended computing power node, wherein the expected scheduling resources are predicted by a preset intelligent prediction model; based on the actual scheduling resources and expected scheduling resources of each historically recommended computing power node, determine the initial scheduling value of computing power resources corresponding to each historically recommended computing power node; compare the initial resource scheduling value corresponding to each historically recommended computing power node with the set initial resource scheduling value; if the initial resource scheduling value corresponding to each historically recommended computing power node is higher than or equal to the set initial resource scheduling value, then the corresponding historically recommended node is used as the historically recommended computing power node for the required resource scheduling final value; if the initial resource scheduling value corresponding to each historically recommended computing power node is lower than the set initial resource scheduling value, then the corresponding historically recommended computing power node is used as the historically recommended computing power node for limiting the resource scheduling final value; The scheduling resource calculation module is used to input the final scheduling value of each demand resource and the final scheduling value of the restricted resource into the intelligent prediction model for prediction, and obtain the schedulable resources corresponding to each historically recommended computing power node; the difference between the schedulable resources corresponding to each historically recommended computing power node and the resource scheduling initial value corresponding to the corresponding historically recommended computing power node is calculated to obtain the resource scheduling final value of the corresponding historically recommended computing power node.

[0097] Since the embodiments of the system part correspond to the embodiments of the above-mentioned method, please refer to the above-mentioned method embodiments for the introduction of the computing power resource scheduling system provided by the present invention. The present invention will not be repeated here, and it has the same beneficial effects as the above-mentioned computing power resource scheduling method.

[0098] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the computing resource scheduling method.

[0099] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computing resource scheduling method, characterized in that: The computing resource scheduling method comprises: Obtain the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; The target computing node is selected based on the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; Obtain the historical computing power data of each target computing power node, and input the real-time computing power data and historical computing power data of each target computing power node into the preset neural network model in turn for computing power prediction, and obtain the predicted real-time computing power data of each target computing power node; Based on the real-time computing power data and predicted real-time computing power data of each target computing power node, the working mode of each computing power node is determined respectively, and the working mode includes a stable working mode and a fluctuating working mode; If the target computing power node is in a stable working mode, computing power resources are scheduled according to the historical computing power data of the target computing power node; If the target computing power node is in the fluctuating working mode, computing power resources are scheduled according to the real-time computing power data and predicted real-time computing power data of the target computing power node.

2. The computing resource scheduling method according to claim 1, characterized in that: The matching of computing nodes based on the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated to select the target computing power node includes: According to the node running time in the real-time computing power data of each computing power node and the task execution time of the computing task to be allocated computing power resources, it is judged whether the computing power node matching conditions are met; If the node running time of the computing power node is earlier than the task execution time of the computing task of the computing power resource to be allocated, then determine whether the task execution time of the computing task of the computing power resource to be allocated is between the valley running time and the peak running time of the current computing power node; If the task execution time of the computing task to be allocated computing resources is between the valley running time and the peak running time of the current computing node, the current computing node is determined as a candidate computing node; Feature matching is performed based on the computing power characteristics of each candidate computing power node and the task characteristics of the computing task to be allocated computing power resources, and the candidate computing power node with the highest feature matching degree is selected as the target computing power node.

3. The computing resource scheduling method according to claim 1, characterized in that: If the target computing power node is in the fluctuating working mode, the computing power resource scheduling is performed according to the real-time computing power data and the predicted real-time computing power data of the target computing power node, including: If the target computing power node is in the fluctuating working mode, the numerical difference between the real-time computing power data of the target computing power node and the predicted real-time computing power data is compared in real time; If the numerical difference exceeds the preset difference range, it is determined that the target computing power node currently has a computing power anomaly, and the number of anomalies of the target computing power node is increased by one; When the number of abnormalities accumulated by the target computing power node is within the first preset range, it is determined that the target computing power node is in a regular fluctuation state; when the number of abnormalities accumulated by the target computing power node is within the second preset range, it is determined that the target computing power node is in an irregular fluctuation mode; If the target computing power node is in a state of regular fluctuation, the computing power fluctuation threshold of the target computing power node is determined based on the historical computing power data of the target computing power node, and computing power resources are scheduled according to the computing power fluctuation threshold; If the target computing power node is in an irregular fluctuation state, obtain the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node; Calculate the difference between the latest real-time computing power data and the latest predicted real-time computing power data to obtain the computing power fluctuation range of the target computing power node; Based on the computing power fluctuation amplitude of the target computing power node, determine whether the target computing power node has a computing power shortage and determine the computing power prediction accuracy of the target computing power node in the fluctuating working mode; If the target computing power node has a computing power shortage and the computing power prediction accuracy is high, the type of the target computing power node is marked as a stable computing power shortage node; If the target computing power node has a computing power shortage and the computing power prediction accuracy is low, the type of the target computing power node is marked as a sudden computing power shortage node; Computing resources are scheduled based on the type of target computing node.

4. The computing resource scheduling method according to claim 3, characterized in that: The method of judging whether the target computing power node has a computing power shortage and judging the computing power prediction accuracy of the target computing power node in a fluctuating working mode based on the computing power fluctuation amplitude of the target computing power node includes: If the value of the target computing power node's computing power fluctuation range is a negative number, it is determined that the target computing power node has a computing power shortage. If the value of the target computing power node's computing power fluctuation range is a positive number, it is determined that the target computing power node has no computing power shortage. Obtain the maximum historical computing power fluctuation range among the historical computing power fluctuation ranges of the target computing power node in the fluctuation working mode; When the difference between the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node exceeds the deviation threshold range, the difference between the latest real-time computing power data and the latest predicted real-time computing power data of the target computing power node is calculated to obtain the real-time computing power fluctuation range of the target computing power node; Calculate the ratio of the real-time computing power fluctuation amplitude of the target computing power node to the maximum historical computing power fluctuation amplitude. If the ratio is greater than or equal to the preset ratio threshold, the computing power prediction accuracy of the target computing power node is determined to be high. If the ratio is less than the ratio threshold, the computing power prediction accuracy of the target computing power node is determined to be low.

5. The computing resource scheduling method according to claim 3, characterized in that: The computing resource scheduling based on the type of the target computing node includes: If the type of the target computing power node is a stable missing computing power node, the real-time computing power data and the predicted real-time computing power data of the stable missing computing power node at different times are obtained and regression operations are performed to obtain the stable predicted computing power of the stable missing computing power node within the deviation threshold range; Add the stable predicted computing power and the predicted real-time computing power of the stable missing computing power node to obtain the predicted real-time computing power increase value of the stable missing computing power node, and subtract the stable predicted computing power and the predicted real-time computing power of the stable missing computing power node to obtain the predicted real-time computing power decrease value of the stable missing computing power node; The computing power resources corresponding to the predicted real-time computing power increase value of the stable missing computing power node are dispatched to the stable missing computing power node, and the computing power resources corresponding to the predicted real-time computing power decrease value of the stable missing computing power node are dispatched to other computing power nodes.

6. The computing resource scheduling method according to claim 3, characterized in that: The computing resource scheduling based on the type of the target computing node includes: If the type of the target computing power node is a sudden missing computing power node, the sudden missing computing power node is classified into a first missing computing power node whose computing power missing threshold is lower than a preset computing power threshold and a second missing computing power node whose computing power missing threshold is greater than or equal to the computing power threshold; If the type of the sudden missing computing power node is the first missing computing power node, the adjacent computing power nodes of the first missing computing power node are obtained, the first computing power threshold of the first missing computing power node is predicted according to the real-time computing power data of the adjacent computing power nodes, and the computing power resources in the first missing computing power node are dispatched to the adjacent computing power nodes and the first missing computing power node according to the preset ratio; If the type of the suddenly missing computing power node is the second missing computing power node, the computing power data change of the second missing computing power node within a preset time is monitored in real time; If the real-time computing power data of the second missing computing power node exceeds the first threshold range or the second threshold range, the second missing computing power node is determined to be a sudden change computing power node; if the real-time computing power data of the second missing computing power node is within the first threshold range or the second threshold range, the second missing computing power node is determined to be a continuously reduced computing power node; If the second missing computing power node is a node with a sudden change in computing power, the second missing computing power node will be eliminated. If the second missing computing power node is a node with a continuously reduced computing power, the computing power resources in the second missing computing power node will be dispatched to other computing power nodes according to a preset ratio.

7. The computing resource scheduling method according to claim 1, characterized in that: The neural network model is obtained by the following training method: Obtain the historical real-time computing power data and historical predicted computing power data of each computing power node, where the historical computing power data of each period includes the error characteristic parameters of the actual computing power value and the predicted computing power value corresponding to the timestamp; Constructing an initial neural network model, wherein the input layer of the initial neural network model inputs historical real-time computing power data and historical predicted computing power data, and the output layer outputs the current predicted real-time computing power data predicted by the computing power node; According to the historical real-time computing power data and the historical predicted computing power data of each computing power node, a loss function of the initial neural network model is constructed, and the initial neural network model is trained to obtain a trained neural network model; Among them, the hidden layer input of the initial neural network model is as follows: in, Represents the historical real-time computing power data vector of the kth input of the hidden layer computing power node i, Represents the historical predicted computing power data vector of the k-th input of the hidden layer computing power node i, is the historical real-time computing power data of the kth input of computing power node i, is the input weight of the historical real-time computing power data of the kth input of computing power node i, is the bias term of computing power node i, is the input weight coefficient of the kth input, N is the number of neurons in the input layer, f is the activation function, is the historical predicted computing power data of the kth input of computing power node i, is the input weight of the historical predicted computing power data of the kth input of computing power node i, is the bias term of computing power node i, is the input weight coefficient of the kth input, and M is the number of neurons in the input layer; Among them, the loss function of the initial neural network model is as follows: in, represents the loss function of the initial neural network model, is the computing power data distribution matrix of computing power node i and computing power node j in the historical time period, S is the number of computing power node samples, is the real-time computing power data of computing power node j, Represents the predicted computing power data of computing power node j predicted by the model.

8. The computing resource scheduling method according to any one of claims 1 to 7, characterized in that: The computing resource scheduling method further includes: Obtain the average resource utilization rate and maximum resource utilization rate of each target computing power node in each period of the historical cycle; Subtract the maximum resource utilization rate of each target computing power node in each period of the historical cycle from the average resource utilization rate to obtain the resource utilization rate difference of each target computing power node in each period of the historical cycle; Multiply the computing power resources corresponding to each time period of each target computing power node in the historical cycle by the difference in resource utilization rate in the corresponding time period to obtain the computing power resource utilization rate corresponding to each time period of each target computing power node in the historical cycle; Compare the computing resource utilization rate of each target computing power node in each period of the historical cycle with the set resource utilization rate. If the computing resource utilization rate of each target computing power node in each period of the historical cycle is higher than or equal to the set resource utilization rate, the corresponding target computing power node will be used as the historical recommended computing power node for the next computing power resource scheduling; Obtain the actual scheduling resources and expected scheduling resources of each historically recommended computing power node, where the expected scheduling resources are predicted by a preset intelligent prediction model; Based on the actual scheduling resources and expected scheduling resources of each historically recommended computing power node, determine the initial scheduling value of computing power resources corresponding to each historically recommended computing power node; Compare the resource scheduling initial value corresponding to each historically recommended computing power node with the set resource scheduling initial value. If the resource scheduling initial value corresponding to each historically recommended computing power node is higher than or equal to the set resource scheduling initial value, the corresponding historically recommended node will be used as the historically recommended computing power node for the required resource scheduling final value; if the resource scheduling initial value corresponding to each historically recommended computing power node is lower than the set resource scheduling initial value, the corresponding historically recommended computing power node will be used as the historically recommended computing power node for limiting the resource scheduling final value; Inputting the final scheduling value of each demand resource and the final scheduling value of the restricted resource into the intelligent prediction model for prediction, respectively, to obtain the schedulable resources corresponding to each historically recommended computing power node; The difference between the schedulable resources corresponding to each historically recommended computing power node and the initial resource scheduling value corresponding to the corresponding historically recommended computing power node is calculated to obtain the final resource scheduling value of the corresponding historically recommended computing power node.

9. A computing resource scheduling system, characterized in that: The computing resource scheduling system includes: The acquisition module is used to obtain the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated; A matching module is used to match computing nodes based on the real-time computing power data of each computing power node and the computing tasks of the computing power resources to be allocated, so as to select the target computing power node; The prediction module is used to obtain the historical computing power data of each target computing power node, and input the real-time computing power data and historical computing power data of each target computing power node into the preset neural network model in turn to perform computing power prediction, so as to obtain the predicted real-time computing power data of each target computing power node; A judgment module, used to judge the working mode of each computing power node based on the real-time computing power data and the predicted real-time computing power data of each target computing power node, wherein the working mode includes a stable working mode and a fluctuating working mode; The scheduling module is used to schedule computing power resources according to the historical computing power data of the target computing power node if the target computing power node is in a stable working mode; if the target computing power node is in a fluctuating working mode, the computing power resources are scheduled according to the real-time computing power data and predicted real-time computing power data of the target computing power node.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the steps of the computing resource scheduling method described in any one of claims 1 to 8.

Images