Calculation power distribution processing method and device

Abstract

The embodiment of the invention provides a computing power distribution processing method and device, and the method comprises the steps: predicting first demand data in target time in a data processing process, and predicting the first demand data in the target time, computing node matching is carried out in all the computing nodes according to the first demand data and the computing power information of all the computing nodes, and if the matching result is that no computing node matched with the first demand data exists, the first computing node with the computing power configuration information matched with the first demand data is screened out from all the computing nodes; and calculating the second demand data corresponding to the to-be-processed task in the first computing node, predicting the processing time of the second computing node for the to-be-processed task in each computing node according to the second demand data, and migrating the to-be-processed task from the first computing node to the target computing node corresponding to the target processing time, thereby improving the data processing efficiency. And the reliability of computing power distribution is improved.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a computing power allocation processing method and device. Background Art

[0002] With the continuous development of Internet technology and the increasing demand for computing, more and more users need devices with stronger computing power and faster processing speed for data processing. Some institutions can build computing power networks to provide computing power support to all parties. Users from all parties can use the computing power networks provided by the institutions for data processing, saving a lot of costs. However, the computing power networks provided by the institutions also have the problem of uneven computing power distribution and mismatch with user needs. In the existing technology, computing power networks usually schedule and allocate computing power for computing power tasks. It is difficult to handle large-scale computing power tasks. How to enable computing power networks to better provide computing power support to users has become the focus of attention of all parties. Summary of the invention

[0003] The purpose of one embodiment of the present specification is to provide a computing power allocation processing method and device to solve the problem of low computing efficiency caused by poor computing power allocation during the computing power allocation processing.

[0004] To solve the above technical problems, an embodiment of this specification is implemented as follows: In a first aspect, an embodiment of the present specification provides a computing power allocation processing method, including: Predicting first demand data within a target time, and matching computing nodes among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, select a first computing node whose computing power configuration information matches the first demand data from among the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; Predicting, according to the second demand data, a processing time of a second computing node among the computing nodes for the task to be processed; Migrate the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time.

[0005] In a second aspect, another embodiment of the present specification provides a computing power allocation processing device, including: A first node matching module, used to predict first demand data within a target time, and perform computing node matching among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, running a second node matching module, the second node matching module is used to screen out a first computing node whose computing power configuration information matches the first demand data from the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; a processing time prediction module, configured to predict, according to the second demand data, a processing time of a second computing node among the computing nodes for the task to be processed; The task migration module is used to migrate the to-be-processed task from the first computing node to a target computing node corresponding to a target processing time.

[0006] In the third aspect, another embodiment of the present specification provides a computing power allocation processing device, including: a memory, a processor, and computer executable instructions stored on the memory and executable on the processor, and when the computer executable instructions are executed by the processor, the steps of the computing power allocation processing method described in the first aspect above are implemented.

[0007] In a fourth aspect, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the computing power allocation processing method described in the first aspect above are implemented.

[0008] In a fifth aspect, another embodiment of the present specification provides a computer program product, wherein the computer program product includes a computing power allocation processing program, and when the computing power allocation processing program is executed by a processor, the steps of the computing power allocation processing method described in the first aspect above are implemented.

[0009] The computing power allocation processing method provided in this embodiment predicts the first demand data within the target time during the data processing process, and predicts the computing tasks that will be received within the target time in advance by predicting the demand. The effectiveness of computing power allocation is improved by predicting in advance, and then computing nodes are matched in each computing node according to the first demand data and the computing power information in each computing node. If the matching result is that there is no computing node matching the first demand data, the first computing node whose computing power configuration information matches the first demand data is screened out in each computing node, and the first computing node that meets the demand is screened out by the computing power configuration information and the first demand data, so as to migrate the computing nodes received subsequently. The method frees up computing power for computing tasks, thereby improving the accuracy of computing power allocation and data processing efficiency, and calculates the second demand data corresponding to the task to be processed in the first computing node, and predicts the processing time of the second computing node in each computing node for the task to be processed according to the second demand data. By calculating the processing time of each computing node, the computing node with the highest processing efficiency is selected for the task to be processed to process the task, thereby improving the accuracy of computing power allocation. Finally, the task to be processed is migrated from the first computing node to the target computing node corresponding to the target processing time, thereby freeing up space on the adapted first computing node for the task to be allocated at the target time, thereby improving data processing efficiency and further improving the high availability of the computing platform and / or computing power network. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate one or more embodiments of the present specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. Figure 1 A processing flow chart of a computing power allocation processing method provided for one or more embodiments of this specification; Figure 2 A processing flow chart of a computing power allocation processing method applied to a computing power allocation scenario provided by one or more embodiments of this specification; Figure 3 A schematic diagram of a computing power allocation processing device provided for one or more embodiments of this specification; Figure 4 A schematic diagram of the structure of a computing power allocation processing device provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0011] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0012] This specification provides an embodiment of a computing power allocation processing method: Reference Figure 1 , which shows a processing flow chart of a computing power allocation processing method provided in this embodiment, referring to Figure 2 , which shows a processing flow chart of a computing power allocation processing method provided in this embodiment and applied to a computing power allocation scenario.

[0013] Reference Figure 1 The computing power allocation processing method provided in this embodiment specifically includes the following steps S102 to S108.

[0014] Step S102: predicting first demand data within a target time, and performing computing node matching among the computing nodes according to the first demand data and computing power information of the computing nodes.

[0015] In this embodiment, the demand data is data used to characterize the computing resources required to process the computing task. Specifically, the demand data can be the node type of the computing node adapted to the computing task, or the remaining computing power and / or remaining space of the computing node required for the computing task, such as the storage space of the computing node, or the instantaneous computing power and / or computing power configuration of the computing node required for the computing task, such as the number of bytes processed by the computing node per second. Optionally, the demand data includes: the demand node type, computing power demand information and / or configuration demand information. In addition, the demand data can also be the memory data of the computing node, the configuration data of the computing node, the usage scenario data of the computing node, the processing delay and / or processing efficiency of the computing node.

[0016] In specific implementation, the target time of external input is obtained, the first demand data corresponding to the task to be assigned within the target time is predicted, and the computing node matching is performed in each computing node according to the first demand data and the computing power information of each computing node of the computing power network and / or computing platform. If the matching result is that there is a computing node matching the first demand data, no processing is performed. If the matching result is that there is no computing node matching the first demand data, step S104 is executed. Specifically, the task to be assigned at the target time and / or the predicted time can be predicted first, that is, the algorithm task that may be received within the target time, and then the first demand data of the task to be assigned can be predicted.

[0017] In this embodiment, the computing power information is configuration information used to characterize the computing power of the computing node. Specifically, the computing power information can be the storage space and / or remaining computing power of the computing node, that is, the maximum storage capacity of the computing power task that can be received. The computing power information can also be the computing power configuration of the computing node, that is, the number of bytes processed by the computing node per unit time. The computing power information can also be the node type of the computing node, for example, the node type of the computing node that processes images is the image processing type. Optionally, the computing power information includes: remaining computing power information, computing power configuration information and / or node type.

[0018] During the specific execution process, in order to improve data processing efficiency and enhance the prediction accuracy of the first demand data, the first demand data may be predicted and processed through a pre-trained model; in an optional implementation manner provided in this embodiment, predicting the first demand data within the target time includes: Inputting the target time into a prediction model to perform task prediction and obtain tasks to be assigned; Inputting the task to be assigned into a demand calculation model to calculate computing power demand, and obtaining the first demand data; The calculating of the second demand data corresponding to the task to be processed in the first computing node is performed by using the demand calculation model.

[0019] Specifically, two pre-trained models can be used to predict the tasks to be assigned and the demand data respectively. The target time and / or preset time can be input into the prediction model to perform task prediction to obtain the tasks to be assigned that may be received at the target time and / or preset time. The tasks to be assigned can be input into the demand calculation model to perform demand calculation to obtain the second demand data corresponding to the tasks to be assigned.

[0020] In addition, the tasks to be assigned and / or the first demand data may not be predicted through the model, but may be predicted and processed through the task prediction algorithm and / or the demand prediction algorithm to save the model training cost.

[0021] In the specific task prediction process, task prediction can be performed based on the historical computing power data of the computing power network and / or the computing platform, that is, the historical tasks received in the historical time. The real historical data can be used as the basis for prediction, thereby improving the accuracy and reliability of task prediction. In an optional implementation manner provided in this embodiment, task prediction includes: Filter out the historical computing power data to find the historical task types and historical data volumes of the historical computing power tasks within the target time; The task to be assigned is constructed according to the historical task type and the historical data volume.

[0022] In this embodiment, the historical computing task may be a periodic computing task, such as an inspection task, which may specifically be an image inspection task, a text inspection task, or a table inspection task.

[0023] In the specific demand calculation process, the demand node type of the computing node adapted to the computing task can be determined according to the task type of the computing task, and the remaining computing power and / or storage space required for the computing task and the computing power configuration required for the computing task can be calculated according to the data volume of the computing task to improve the reliability of the demand data; in an optional implementation manner provided in this embodiment, the computing power demand calculation includes: Extracting the task type and data volume of the task to be assigned, and querying the required node type of the computing node matching the task type; Calculate computing power requirement information and configuration requirement information that are adapted to the data volume, and determine the required node type, the computing power requirement information, and the configuration requirement information as the first demand data.

[0024] It should be noted that, in this embodiment, the task to be assigned can be a task for one user or a task for multiple users; for multiple tasks to be processed for multiple users, each task to be processed can perform demand data prediction and subsequent computing node matching process according to the above-mentioned prediction process. The specific execution process refers to the above-mentioned and following execution processes, and this embodiment will not be repeated here.

[0025] Correspondingly, for multiple tasks to be processed by multiple users, computing power requirements can also be calculated based on the user data of the users to improve the accuracy of the demand data. Specifically, the user data can be the user's identity data and / or the computing power demand data input by the user.

[0026] It should also be noted that the prediction model in this embodiment can be a model embedded with multiple sub-models and / or modules. Specifically, the prediction model may include a task prediction sub-model for task prediction, the prediction model may also include a node matching sub-model for node matching, and the prediction model may also include a time prediction sub-model for time prediction. Optionally, the prediction model includes: a task prediction sub-model, a node matching sub-model and / or a time prediction sub-model. In addition, the demand calculation model can also be embedded in the prediction model as a sub-model and / or module, that is: the target time can be input into the prediction model for task prediction and demand calculation.

[0027] The training of the task prediction sub-model can be to use the preset time as a sample, the historical computing power tasks as labels, input the preset time into the task prediction sub-model, output the calculation result, compare the calculation result with the label and modify the task prediction sub-model according to the comparison result. The historical computing power tasks used as labels can be pre-set or real computing power tasks can be obtained. And so on. The task prediction sub-model is trained using the above-mentioned task prediction sub-model training method until the difference between the computing power tasks predicted by the task prediction sub-model and the historical computing power tasks used as labels converges. The training of the task prediction sub-model is then completed. The training of the task prediction sub-model can be performed on a cloud server or offline.

[0028] For training of a specific demand calculation model, sample tasks may be used as sample data, and historical demand data corresponding to the sample tasks may be used as labels to construct a sample data set. The demand calculation model may be trained with real data from a computing network and / or computing platform, thereby improving the reliability of the demand calculation model. In an optional implementation manner provided in this embodiment, the demand calculation model may be obtained by training in the following manner: Using sample tasks as sample data and using historical demand data corresponding to the sample tasks as labels; Inputting the sample data into the demand calculation model to be trained to calculate the computing power demand, and obtaining the demand data corresponding to the sample task; The demand calculation model to be trained is parameterized according to the loss calculated between the label and the demand data to obtain the demand calculation model.

[0029] In the specific computing node matching process, the computing nodes of the computing power network and / or the computing platform may be matched according to the computing power information and the first demand data, and the computing nodes matching the first demand data may be screened out; in this regard, the computing nodes may be matched by a pre-trained prediction model to improve the effectiveness and accuracy of node matching; in an optional implementation manner provided in this embodiment, computing node matching may be performed, including: The first demand data and the computing power information of each computing node are input into a prediction model to perform node matching to obtain a node matching result.

[0030] Specifically, the first demand data and computing power information can be input into the node matching submodel in the prediction model for node matching processing. If the matching result is that there is no computing node matching the first demand data, it indicates that there is no computing node matching the first demand data in the computing power network and / or computing nodes. If the matching result is that there is a computing node matching the first demand data, it indicates that there is a computing node matching the first demand data in the computing power network and / or computing nodes, and corresponding processing can be performed.

[0031] In the specific node matching process, the demand information and / or the demand node type may be configured according to the computing power demand information in the demand data to perform node matching in each computing node, so as to improve the matching accuracy and availability of the computing nodes; in an optional implementation manner provided in this embodiment, the node matching includes: Extracting computing power requirement information and configuration requirement information included in the first requirement data, and detecting whether there is a first candidate node in the computing power information whose computing power configuration information matches the configuration requirement information; If not present, the test result is determined to be non-existent; If so, detect whether there is a computing node in the computing power information of the first candidate node whose remaining computing power information matches the computing power requirement information, and use the detection result as the node matching result.

[0032] The training of the node matching sub-model can be to use the sample demand data as the sample, the preset computing node as the label, input the sample demand data into the node matching sub-model, output the calculation result, compare the calculation result with the label and modify the node matching sub-model according to the comparison result. The preset computing node as the label can be a pre-set one or a real computing node can be obtained. And so on. The node matching sub-model is trained using the above-mentioned training method until the difference between the computing node matched by the node matching sub-model and the preset computing node as the label converges. Then the training of the node matching sub-model is completed. The node matching sub-model can be trained on a cloud server or offline.

[0033] In addition, there is a case where there is a computing node matching the first demand data in the computing nodes of the computing power network and / or the computing platform. In this case, a computing node with higher processing efficiency can be selected from the multiple computing nodes obtained by matching to improve data processing efficiency. In an optional implementation manner provided by this embodiment, if the matching result is that there is a computing node matching the first demand data, it also includes: querying computing power configuration information of each third computing node matching the first demand data, and calculating a first processing time of each third computing node for a task to be assigned corresponding to the first demand data based on the computing power configuration information; The third computing nodes are prioritized according to the first processing time, so as to allocate the to-be-allocated tasks to the computing nodes corresponding to the target priorities at the target time.

[0034] It should be noted that the selection operation of the computing node corresponding to the target priority can also be implemented through a pre-trained neural network model. Specifically, the first demand data and the computing power configuration information of each third computing node can be input into the neural network model for node matching to obtain the computing node that processes the task to be assigned.

[0035] It should also be noted that the selection operation of the computing node corresponding to the target priority can also be implemented through the prediction model. Specifically, the first demand data and the computing power information of each computing node can be input into the prediction model, and the computing node matching is first performed. If the matching result is that there is a computing node, a secondary computing node matching is performed to obtain the computing node that processes the task to be assigned.

[0036] Step S104, selecting a first computing node whose computing power configuration information matches the first demand data from among the computing nodes, and calculating second demand data corresponding to the task to be processed in the first computing node.

[0037] This embodiment reserves computing nodes that meet the requirements of the tasks to be assigned in advance by predicting the demand data of the tasks to be assigned within the target time, and allocates computing power through simulation prediction, thereby improving the utilization of computing power, improving data processing efficiency, and improving the high availability of the computing power network and / or computing platform.

[0038] In specific implementation, the computing power configuration information of each computing node in the computing power network and / or computing platform is queried, and the first computing node whose computing power configuration information matches the first demand data is screened out from each computing node, and the tasks to be processed in the first computing node are extracted and the second demand data corresponding to the tasks to be processed is calculated. Optionally, the first computing node is obtained by node matching through a prediction model.

[0039] During the specific execution process, the first computing node can be screened through the above-mentioned prediction model to improve the accuracy and reliability of the first computing node; in an optional implementation provided by this embodiment, the first computing node is screened in each computing node, including: inputting the matching result into the prediction model for secondary matching to obtain the first computing node.

[0040] During the specific execution process, in order to improve the accuracy and reliability of the second demand data, the above-mentioned demand calculation model can also be used to perform demand calculation; in an optional implementation provided by this embodiment, the second demand data is obtained by performing demand calculation using the demand calculation model.

[0041] It should be noted that the operations of matching computing nodes according to the computing power information in each computing node and the first demand data and the operation of screening out the first computing node from each computing node can be performed through the prediction model. Specifically, the first demand data and the computing power information of each computing node can be input into the prediction model, and the prediction model first performs node matching to obtain a matching result of non-existence, and then performs node matching according to the first demand data and the computing power configuration information to obtain the first computing node; that is, steps S102 to S104 can be replaced by: predicting the first demand data within the target time, inputting the first demand data and the computing power information of each computing node into the prediction model for node matching, and outputting the computing node whose computing power information matches the first demand data or the first computing power node whose computing power configuration information matches the first demand data according to the matching result.

[0042] In addition, the first computing node may be screened by a screening algorithm to save model training costs. In an optional implementation provided by this embodiment, screening the first computing node from each computing node includes: Filtering out candidate nodes from among the computing nodes, the node types of which match the required node types in the first requirement data; The computing power configuration information of each candidate node is queried, and a computing node whose computing power configuration information matches the configuration requirement information in the first requirement data is selected from the candidate nodes as the first computing node.

[0043] Step S106: predicting the processing time of the second computing node among the computing nodes for the task to be processed according to the second demand data.

[0044] During specific implementation, the processing time required for each second computing node to process the task to be processed is calculated based on the data volume in the second demand data corresponding to the task to be processed and the computing power configuration information of the second computing node.

[0045] In the specific execution process, time prediction processing can be performed through the prediction model to improve the accuracy and reliability of time prediction; in an optional implementation manner provided by this embodiment, the processing time of the task to be processed is calculated, including: The second demand data and the computing power information of the second computing node are input into a prediction model to predict the processing time and obtain the processing time.

[0046] Specifically, the processing time can be calculated based on the computing power configuration information of the second computing node; in an optional implementation provided by this embodiment, the processing time prediction includes: extracting the data volume in the second demand data, performing time calculation based on the computing power configuration information and the data volume in the computing power information of the second computing node, and obtaining the processing time.

[0047] It should be noted that the processing time prediction can be performed through the time prediction sub-model and / or time prediction module in the prediction model; the training of the time prediction sub-model can be to use the sample computing power information as the sample, the preset processing time as the label, the sample computing power information as the input node matching sub-model, output the calculation result, compare the calculation result with the label and modify the time prediction sub-model according to the comparison result. The preset processing time as the label can be pre-set or the real computing node can be obtained. Similarly, the time prediction sub-model is trained using the above-mentioned training method of the time prediction sub-model until the difference between the processing time predicted by the time prediction sub-model and the preset processing time as the label converges. The training of the time prediction sub-model is then completed. The training of the time prediction sub-model can be performed on a cloud server or offline.

[0048] In practical applications, there is also a situation where the individual computing power information of each second computing node does not match the second demand data, that is, the processing time is greater than the preset time threshold, for example, when the remaining computing power information and / or computing power configuration information of the second computing power node does not match the second demand data, the processing time obtained is a maximum value greater than the time threshold; in view of this, the task to be processed can be split into multiple subtasks and respectively assigned to each second computing node for processing, thereby improving data processing efficiency and improving the high availability of the computing power network and / or computing platform; in an optional implementation manner provided by this embodiment, after predicting the processing time of the task to be processed, it also includes: If the processing time is greater than the processing time threshold corresponding to the task to be processed, input the second demand data and the computing power information of the second computing node into a neural network model and / or a task matching model for task splitting and node matching; A plurality of computing nodes matching the second demand data are obtained, so as to split the tasks to be processed and distribute them to the plurality of computing nodes for task processing.

[0049] Specifically, when the calculation result of each second computing node for the data to be processed is greater than the processing time threshold, the second demand data and the computing power information of the second computing node are input into the task matching model for task splitting and node matching, and multiple computing nodes matching the second demand data are obtained, so that the tasks to be processed can be split and respectively assigned to the multiple computing nodes for task processing.

[0050] For example, when the calculation result of each second computing node for the data to be processed is greater than the processing time threshold, the task to be processed is split into multiple sub-tasks, and the processing time under multiple allocation combinations is calculated according to the data amount of the multiple sub-tasks and the computing power configuration information of the second computing node. The split result with a smaller processing time and the node matching result are selected as the matching result, so that the task to be processed can be split according to the split result and then allocated to multiple matching computing nodes.

[0051] Step S108: Migrate the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time.

[0052] In specific implementation, the target computing node corresponding to the target processing time can be first selected in the second computing node, and the task to be processed can be migrated from the first computing node to the target computing node so that the target computing node processes the task to be processed and the first computing node processes the task to be assigned corresponding to the first demand data within the target time.

[0053] In actual applications, there are also cases where the simulation / prediction results are inconsistent with the actual processing results, for example, the predicted processing time is different from the actual processing time. In this regard, the difference between the predicted processing time and the actual processing time of the computing node can be obtained. If the difference is greater than a threshold, it indicates that the computing node has a fault, so as to improve the high availability of the computing network and / or computing platform. In an optional implementation manner provided by this embodiment, after migrating the task to be processed from the first computing node to the target computing node, it also includes: Obtaining the status information of the target computing node within the target time; Calculate the difference between the computing time of the task to be processed in the status information and the target processing time, detect whether the difference is greater than a threshold, and if so, mark the target computing node as a faulty node; If not, no action is required.

[0054] In addition, for the to-be-allocated task corresponding to the first demand node, the processing time of the first computing node for the to-be-allocated task can also be obtained, and it is detected whether the processing time is greater than a preset processing time threshold. If the processing time is greater than the threshold, it indicates that the computing node has a fault, so as to improve the high availability of the computing power network and / or the computing platform; in an optional implementation manner provided by this embodiment, after migrating the to-be-processed task from the first computing node to the target computing node, it also includes: Acquire the to-be-allocated tasks corresponding to the first demand data, and allocate the to-be-allocated tasks to the first computing node for task processing; Detecting whether the processing time of the task to be assigned is greater than a threshold, and if so, marking the first computing node as a faulty node; If not, no action is required.

[0055] In addition, in actual applications, there is a situation where the computing power tasks received at the target time do not match the predicted computing power tasks. In response to this, computing power allocation, computing node matching and migration of tasks to be processed can be performed for the computing power tasks actually received. The specific operations can be performed with reference to the above-mentioned execution process, and this embodiment will not be repeated again.

[0056] To summarize, the one or more computing power allocation processing methods provided in the present embodiment predict the first demand data within the target time during the data processing process, and predict the computing tasks to be received within the target time in advance by predicting the demand, thereby improving the effectiveness of computing power allocation by predicting in advance, and then matching computing nodes in each computing node according to the first demand data and the computing power information in each computing node. If the matching result is that there is no computing node matching the first demand data, the first computing node whose computing power configuration information matches the first demand data is screened out in each computing node, and the first computing node that meets the demand is screened out by jointly using the computing power configuration information and the first demand data, so as to migrate the task for the subsequent computing node. The computing tasks to be processed are subsequently freed up for computing power, thereby improving the accuracy of computing power allocation and data processing efficiency, and calculating the second demand data corresponding to the tasks to be processed in the first computing node, and predicting the processing time of the second computing node in each computing node for the tasks to be processed according to the second demand data. By calculating the processing time of each computing node, the computing node with the highest processing efficiency is selected for the tasks to be processed, thereby improving the accuracy of computing power allocation. Finally, the tasks to be processed are migrated from the first computing node to the target computing node corresponding to the target processing time, thereby freeing up space on the adapted first computing node for the tasks to be allocated at the target time, thereby improving data processing efficiency and further improving the high availability of the computing platform and / or computing power network.

[0057] The following combination Figure 2 , taking the application of the computing power allocation processing method provided in this embodiment in the computing power allocation scenario as an example, the computing power allocation processing method provided in this embodiment is further explained. Figure 2 The computing power allocation processing method applied to the computing power allocation scenario specifically includes steps S202 to S216.

[0058] Step S202: input the target time into the prediction model to perform task prediction to obtain the tasks to be assigned.

[0059] Optionally, the prediction model includes a task prediction sub-model.

[0060] Step S204, performing computing power demand calculation on the demand calculation model of the task to be assigned to obtain first demand data.

[0061] Step S206, inputting the first demand data and the computing power information of each computing node into the prediction model to perform node matching and obtain a node matching result; If the matching result is that the computing node exists in the first requirement data matching, execute step S208; If the matching result is that the computing node does not exist in the first requirement data matching, step S210 to step S216 are executed.

[0062] Optionally, the prediction model includes a node matching sub-model.

[0063] Step S208: Input the first demand data and the computing power configuration information of the third computing node into the neural network model to perform node screening to obtain a computing node that processes the task to be assigned.

[0064] Step S210: input the first demand information and the computing power configuration information of each computing node into the prediction model for secondary matching to obtain the first computing node.

[0065] Step S212: Input the task to be processed in the first computing node into the demand calculation model to calculate the computing power demand and obtain second demand data.

[0066] Step S214: input the second demand data and the computing power information of the second computing node into the prediction model for time prediction to obtain the processing time of the second computing node for the task to be processed.

[0067] Optionally, the prediction model includes a time prediction sub-model.

[0068] Step S216: Migrate the task to be processed from the first computing node to the target computing node corresponding to the target processing time.

[0069] Optionally, after step S216 is executed, it also includes: obtaining status information of the target computing node within the target time, calculating the difference between the computing time of the task to be processed and the target processing time in the computing status information, detecting whether the difference is greater than a threshold, if so, marking the target computing node as a faulty node; if not, no processing is performed.

[0070] It should be noted that any one of steps S202 to S216 or any combination of multiple steps can be selected from steps S102 to S108 to form a new implementation method according to the needs of implementation deployment; and any one or multiple technical features in the technical scheme composed of steps S202 to S216 can be selected from steps S102 to S108 to form a new implementation method according to the needs of actual deployment, or the technical features in one or more optional implementation methods provided by steps S102 to S108 can be selected to combine into a new implementation method, which will not be elaborated here one by one.

[0071] Figure 3 A schematic diagram of a computing power allocation processing device provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, the device comprises: A first node matching module 302, used to predict first demand data within a target time, and perform computing node matching among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, the second node matching module 304 is run, and the second node matching module 304 is used to screen out the first computing node whose computing power configuration information matches the first demand data from the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; A processing time prediction module 306, configured to predict the processing time of the second computing node among the computing nodes for the task to be processed according to the second demand data; The task migration module 308 is used to migrate the to-be-processed task from the first computing node to a target computing node corresponding to a target processing time.

[0072] The computing power allocation processing device provided in this embodiment, during the data processing process, first runs the first node matching module 302 to predict the first demand data within the target time, and predicts the computing tasks to be received within the target time in advance by predicting the demand, thereby improving the effectiveness of computing power allocation by predicting in advance, and then performs computing node matching in each computing node according to the first demand data and the computing power information in each computing node. If the matching result is that there is no computing node matching the first demand data, runs the second node matching module 304 to screen out the first computing node whose computing power configuration information matches the first demand data in each computing node, and jointly screens out the first computing node that meets the demand by using the computing power configuration information and the first demand data, so as to migrate the task. Computing power is freed up for the computing tasks received subsequently, improving the accuracy of computing power allocation and data processing efficiency, and calculating the second demand data corresponding to the task to be processed in the first computing node. The processing time prediction module 306 is run to predict the processing time of the second computing node in each computing node for the task to be processed according to the second demand data. By calculating the processing time of each computing node, the computing node with the highest processing efficiency is selected for the task to be processed, thereby improving the accuracy of computing power allocation. Finally, the task migration module 308 is run to migrate the task to be processed from the first computing node to the target computing node corresponding to the target processing time, freeing up space on the adapted first computing node for the task to be allocated at the target time, thereby improving data processing efficiency and further improving the high availability of the computing platform and / or computing power network.

[0073] The computing power allocation processing device provided in one embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0074] Furthermore, an embodiment of the present specification also provides a computing power allocation processing device, Figure 4 A schematic diagram of a computing power allocation processing device provided in an embodiment of this specification is shown in FIG. Figure 4 As shown, the device includes: a memory 401, a processor 402, a bus 403 and a communication interface 404. The memory 401, the processor 402 and the communication interface 404 communicate through the bus 403, and the communication interface 404 may include an input and output interface, which includes but is not limited to a keyboard, a mouse, a display, a microphone, a loudspeaker, etc.

[0075] Figure 4 In the embodiment, the memory 401 stores computer executable instructions that can be run on the processor 402. When the computer executable instructions are executed by the processor 402, the following process is implemented: Predicting first demand data within a target time, and matching computing nodes among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, select a first computing node whose computing power configuration information matches the first demand data from among the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; Predicting, according to the second demand data, a processing time of a second computing node among the computing nodes for the task to be processed; Migrate the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time.

[0076] The computing power allocation processing device provided in this embodiment, through the cooperation of the memory 401, the processor 402, the bus 403 and the communication interface 404, predicts the first demand data within the target time during the data processing process, and predicts the computing tasks to be received within the target time in advance by predicting the demand, thereby improving the effectiveness of computing power allocation by predicting in advance, and then performs computing node matching in each computing node according to the first demand data and the computing power information in each computing node. If the matching result is that there is no computing node matching the first demand data, the first computing node whose computing power configuration information matches the first demand data is screened out in each computing node, and the first computing node that meets the demand is screened out by the computing power configuration information and the first demand data. By migrating tasks, computing power is freed up for subsequently received computing tasks, thereby improving the accuracy of computing power allocation and data processing efficiency, and calculating the second demand data corresponding to the task to be processed in the first computing node, and predicting the processing time of the second computing node in each computing node for the task to be processed based on the second demand data. By calculating the processing time of each computing node, the computing node with the highest processing efficiency is selected for the task to be processed, thereby improving the accuracy of computing power allocation. Finally, the task to be processed is migrated from the first computing node to the target computing node corresponding to the target processing time, thereby freeing up space on the adapted first computing node for the task to be allocated at the target time, thereby improving data processing efficiency and thereby improving the high availability of the computing platform and / or computing power network.

[0077] The computing power allocation processing device provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0078] Furthermore, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented: Predicting first demand data within a target time, and matching computing nodes among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, select a first computing node whose computing power configuration information matches the first demand data from among the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; Predicting, according to the second demand data, a processing time of a second computing node among the computing nodes for the task to be processed; Migrate the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time.

[0079] The computer-readable storage medium provided in the present embodiment predicts the first demand data within the target time during the data processing process, and predicts the computing tasks to be received within the target time in advance by predicting the demand, thereby improving the effectiveness of computing power allocation by predicting in advance, and then matching computing nodes in each computing node according to the first demand data and the computing power information in each computing node. If the matching result is that there is no computing node matching the first demand data, the first computing node whose computing power configuration information matches the first demand data is screened out in each computing node, and the first computing node that meets the demand is screened out by jointly using the computing power configuration information and the first demand data, so as to provide a migration task for the subsequent computing nodes. The computing tasks to be processed are subsequently freed up for computing power, thereby improving the accuracy of computing power allocation and data processing efficiency, and calculating the second demand data corresponding to the tasks to be processed in the first computing node, and predicting the processing time of the second computing node in each computing node for the tasks to be processed according to the second demand data. By calculating the processing time of each computing node, the computing node with the highest processing efficiency is selected for the tasks to be processed, thereby improving the accuracy of computing power allocation. Finally, the tasks to be processed are migrated from the first computing node to the target computing node corresponding to the target processing time, thereby freeing up space on the adapted first computing node for the tasks to be allocated at the target time, thereby improving data processing efficiency and further improving the high availability of the computing platform and / or computing power network.

[0080] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0081] The computer-readable storage medium provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0082] Furthermore, another embodiment of the present specification also provides a computer program product, which implements the following process when executed by a processor: Predicting first demand data within a target time, and matching computing nodes among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, select a first computing node whose computing power configuration information matches the first demand data from among the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; Predicting, according to the second demand data, a processing time of a second computing node among the computing nodes for the task to be processed; Migrate the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time.

[0083] The computer product program provided in this embodiment predicts the first demand data within the target time during the data processing process, and predicts the computing tasks to be received within the target time in advance by predicting the demand, thereby improving the effectiveness of computing power allocation by predicting in advance, and then matching computing nodes in each computing node according to the first demand data and the computing power information in each computing node. If the matching result is that there is no computing node matching the first demand data, the first computing node whose computing power configuration information matches the first demand data is screened out in each computing node, and the first computing node that meets the demand is screened out by jointly using the computing power configuration information and the first demand data, so as to provide a basis for subsequent tasks by migrating tasks. The received computing tasks free up computing power, improve the accuracy of computing power allocation and data processing efficiency, and calculate the second demand data corresponding to the task to be processed in the first computing node, and predict the processing time of the second computing node in each computing node for the task to be processed based on the second demand data. By calculating the processing time of each computing node, the computing node with the highest processing efficiency is selected for the task to be processed to process the task, thereby improving the accuracy of computing power allocation. Finally, the task to be processed is migrated from the first computing node to the target computing node corresponding to the target processing time, freeing up space on the adapted first computing node for the task to be allocated at the target time, thereby improving data processing efficiency and thereby improving the high availability of the computing platform and / or computing power network.

[0084] The computer program product provided in one embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0085] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0088] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0089] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0090] Memory may include non-permanent storage in a computer-readable storage medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable storage medium.

[0091] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0092] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0093] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0094] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A computing power allocation processing method, characterized in that: Applied to a computing platform, the method comprises: Predicting first demand data within a target time, and matching computing nodes among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, select a first computing node whose computing power configuration information matches the first demand data from among the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; Predicting, according to the second demand data, a processing time of a second computing node among the computing nodes for the task to be processed; Migrate the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time.

2. The computing power allocation processing method according to claim 1, characterized in that: The performing computing node matching in each computing node according to the first demand data and the computing power information of each computing node includes: The first demand data and the computing power information of each computing node are input into a prediction model to perform node matching processing to obtain a node matching result.

3. The computing power allocation processing method according to claim 2, characterized in that: The node matching process includes: Extracting computing power requirement information and configuration requirement information included in the first requirement data, and detecting whether there is a first candidate node in the computing power information whose computing power configuration information matches the configuration requirement information; If so, detect whether there is a computing node in the computing power information of the first candidate node whose remaining computing power information matches the computing power requirement information, and use the detection result as the node matching result.

4. The computing power allocation processing method according to claim 2, characterized in that: The first computing node whose computing power configuration information matches the first demand data is selected from the computing nodes by using the prediction model; Correspondingly, the step of selecting a first computing node whose computing power configuration information matches the first demand data from among the computing nodes includes: The node matching result is input into the prediction model for secondary matching to obtain the first computing node.

5. The computing power allocation processing method according to claim 1, characterized in that: The predicting, according to the second demand data, the processing time of the second computing node among the computing nodes for the task to be processed includes: Inputting the second demand data and the computing power information of the second computing node into a prediction model to predict the processing time, thereby obtaining the processing time; The processing time prediction includes: The data volume in the second demand data is extracted, and time calculation is performed according to the computing power configuration information in the computing power information of the second computing node and the data volume to obtain the processing time.

6. The computing power allocation processing method according to claim 1, characterized in that: The first demand data within the predicted target time includes: Inputting the target time into a prediction model to perform task prediction and obtain tasks to be assigned; Inputting the task to be assigned into a demand calculation model to calculate computing power demand, and obtaining the first demand data; The calculating of the second demand data corresponding to the task to be processed in the first computing node is performed by using the demand calculation model.

7. The computing power allocation processing method according to claim 6, characterized in that: The task prediction includes: Filter out the historical computing power data to find the historical task types and historical data volumes of the historical computing power tasks within the target time; The task to be assigned is constructed according to the historical task type and the historical data volume.

8. The computing power allocation processing method according to claim 6, characterized in that: The computing power requirement calculation includes: Extracting the task type and data volume of the task to be assigned, and querying the required node type of the computing node matching the task type; Calculate computing power requirement information and configuration requirement information that are adapted to the data volume, and determine the required node type, the computing power requirement information, and the configuration requirement information as the first demand data.

9. The computing power allocation processing method according to claim 6, characterized in that: The demand calculation model is trained and obtained in the following manner: Using sample tasks as sample data and using historical demand data corresponding to the sample tasks as labels; Inputting the sample data into the demand calculation model to be trained to calculate the computing power demand, and obtaining the demand data corresponding to the sample task; The demand calculation model to be trained is parameterized according to the loss calculated between the label and the demand data to obtain the demand calculation model.

10. The computing power allocation processing method according to claim 1, characterized in that: After matching computing nodes in each computing node according to the first demand data and the computing power information of each computing node, the method further includes: If the matching result is that there are multiple third computing nodes matching the first demand data, query the computing power configuration information of each third computing node, and calculate the first processing time of each third computing node for the to-be-allocated task corresponding to the first demand data based on the computing power configuration information; The third computing nodes are prioritized according to the first processing time, so as to allocate the to-be-allocated tasks to computing nodes corresponding to target priorities at the target time.

11. The computing power allocation processing method according to claim 1, characterized in that: After predicting the processing time of the second computing node among the computing nodes for the task to be processed according to the second demand data, the method further includes: If the processing time is greater than the processing time threshold corresponding to the task to be processed, inputting the second demand data and the computing power information of the second computing node into the task matching model for task splitting and node matching; A plurality of computing nodes matching the second demand data are obtained, so as to split the tasks to be processed and distribute them to the plurality of computing nodes for task processing.

12. The computing power allocation processing method according to claim 1, characterized in that: After migrating the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time, the method further includes: Obtaining the status information of the target computing node within the target time; The difference between the computing time of the task to be processed in the status information and the target processing time is calculated, and whether the difference is greater than a threshold is detected. If so, the target computing node is marked as a faulty node.

13. The computing power allocation processing method according to claim 1, characterized in that: After migrating the to-be-processed task from the first computing node to a target computing node corresponding to the target processing time, the method further includes: Acquire the to-be-allocated tasks corresponding to the first demand data, and allocate the to-be-allocated tasks to the first computing node for task processing; Detect whether the processing time of the task to be assigned is greater than a threshold, and if so, mark the first computing node as a faulty node.

14. The computing power allocation processing method according to claim 1, characterized in that: The step of selecting a first computing node whose computing power configuration information matches the first demand data from among the computing nodes includes: Filtering out candidate nodes from among the computing nodes, the node types of which match the required node types in the first requirement data; The computing power configuration information of each candidate node is queried, and a computing node whose computing power configuration information matches the configuration requirement information in the first requirement data is selected from the candidate nodes as the first computing node.

15. A computing power allocation processing device, characterized in that: The device comprises: A first node matching module, used to predict first demand data within a target time, and perform computing node matching among the computing nodes according to the first demand data and computing power information of each computing node; If the matching result is that there is no computing node matching the first demand data, running a second node matching module, the second node matching module is used to screen out a first computing node whose computing power configuration information matches the first demand data from the computing nodes, and calculate the second demand data corresponding to the task to be processed in the first computing node; a processing time prediction module, configured to predict, according to the second demand data, a processing time of a second computing node among the computing nodes for the task to be processed; The task migration module is used to migrate the to-be-processed task from the first computing node to a target computing node corresponding to a target processing time.

16. A computing power distribution processing device, characterized in that: The device includes a memory and a processor, wherein the memory stores computer executable instructions, and when the computer executable instructions are executed on the processor, the steps of the computing power allocation processing method described in any one of claims 1 to 14 can be implemented.

17. A computer-readable storage medium having computer-executable instructions stored therein, characterized in that: When the computer executable instructions are executed by the processor, the steps of the computing power allocation processing method described in any one of claims 1 to 14 can be implemented.

18. A computer program product, characterized in that The computer program product includes a computing power allocation processing program, and when the computing power allocation processing program is executed by a processor, it can implement the steps of the computing power allocation processing method described in any one of claims 1 to 14 above.

Images