Computing power scheduling method and system

Abstract

This invention relates to a computing power scheduling method. To recommend suitable nodes for user-submitted tasks and reduce the low engagement of clients with limited resources, the computing power scheduling method of this invention includes: receiving a task, wherein the task contains requirements for node performance; determining a currently available set of nodes, the set including a new set of nodes and an existing set of nodes; randomly selecting m nodes from the new set of nodes and adding the m nodes to the set; and selecting q nodes from the existing set of nodes by solving the following formula. t T p u The n nodes with the largest values ​​are added to a set; the set is then output. This invention also relates to a computing power scheduling system and a computer-readable storage medium.

Description

TECHNICAL FIELD

[0001] The application relates to a computing power scheduling method, a computing power scheduling system and a computer readable storage medium. BACKGROUND

[0002] Content-based recommendation is to make recommendations based on the content information of the project, without relying on the evaluation of the project by the user, and more needs to use the method of machine learning to obtain the user's interest information from the case of the feature description about the content. Simply understand is to calculate the correlation between projects according to the attributes of the recommended projects, and then recommend the items with high similarity to the user according to the user's preference record.

[0003] In the content-based recommendation system, the project or object is defined by the attributes of the related features, the system learns the user's interest based on the user's evaluation of the features of the object, and investigates the matching degree of the user's profile and the to-be-predicted project. The user profile model depends on the learning method used, and the commonly used ones are decision tree, neural network and vector-based representation method. The content-based user profile needs the historical data of the user, and the user profile model may change with the change of the user's preference.

[0004] For the computing power scheduling scene participated by multiple nodes, the system needs to run a scheduling algorithm to select multiple reasonable computing power nodes to distribute tasks. Because the bandwidth and computing resources required by the task are not the same, the performance of the nodes is also not the same, and the performance of the nodes may be updated constantly, in addition,

[0005] With the running of the system, the number of nodes gradually decreases from much larger than the number of tasks to less than the number of tasks, so the content-based recommendation algorithm is selected relative to the collaborative filtering recommendation algorithm. SUMMARY

[0006] The application aims to provide a computing power scheduling method, a computing power scheduling system and a computer readable storage medium, which can recommend suitable nodes for the tasks published by users and can reduce the problem of low participation of small resource customers.

[0007] The computing power scheduling method according to the application comprises:

[0008] Step S1: receiving a task, wherein the task contains a demand for node performance;

[0009] Step S2: determining a current available node set, the node set comprising a new node set and an existing node set, wherein the number of times of accepting tasks of the nodes in the new node set is less than a certain threshold, and the number of times of accepting tasks of the nodes in the existing node set is greater than or equal to the certain threshold;

[0010] Step S3: randomly selecting m nodes from the new node set, and adding the m nodes to the set;

[0011] Step S4: selecting q t T p u nodes with the largest values from the existing node set, and adding the n nodes to the set:

[0012]

[0013] where q * and p * are factor vectors, u is a single node, t is a single task, K is a training set, r ut is a label result (0 or 1) of a (u, t) sample in the training set, where 1 represents that the task t is assigned to the node u before, and 0 represents that the task t is not assigned to the node u, q t is a vector associated with the task t generated by q * , and p u is a vector associated with the node u generated by p * , q t T p u represents a predicted value of the node u and the task t;

[0014] Step S5: outputting the set.

[0015] In order to reduce the problem that the participation of small resource customers is low, the application adds a random strategy in a node recommendation algorithm, and selects a plurality of nodes in new nodes with a low number of accepted tasks. Therefore, the node selection algorithm is composed of two parts of heuristic random selection (step S3) and recommendation algorithm recommendation (step S4), which greatly improves the participation rate and activity of the computing power node. Meanwhile, in the recommendation algorithm, a content-based recommendation algorithm is used, which is beneficial to better recommend nodes and make the recommendation result more accurate. Obviously, there is no specific requirement for the order of step S3 and step S4, and the two steps can also be performed in parallel.

[0016] According to a design scheme of the application, the nodes in the node set respectively have performance information, and the performance information includes at least one of the following indexes: bandwidth, computing resource, activity, number of accepted tasks, number of rewards, average completion time, and delay time. For each node, the above indexes in the performance information of the node can evaluate the performance of the node from different dimensions, so that a node meeting the demand of a task for the performance of the node is selected. The above seven indexes play a key role in the node scheduling process compared with other indexes, quantitatively record the characteristics of the node, can quantize the abstract scheduling process, and ensure the fairness and accuracy of the node selection process.

[0017] According to an embodiment of the present application, the received task contains the values of m and n. Therefore, the number of new nodes and existing nodes selected can be determined by the user and input together with the task.

[0018] According to an embodiment of the present application, the formula is solved using a gradient descent algorithm, and the relevant prediction error e ut , q t and p u are updated as follows until convergence:

[0019]

[0020] q t ← q t + γ × (e ut × p u - λ × q t )

[0021] p u ← p u + γ × (e ut × q t - λ × p u )

[0022] wherein the parameter λ is used to control the degree of regularization, and γ represents the learning rate in the gradient algorithm. By solving the formula using the gradient descent algorithm, a reasonable parameter update direction can be selected, the descent direction is stable, and the first n nodes can be ensured to be calculated.

[0023] According to an embodiment of the present application, the performance of each node is represented as N j = {w 1j , w 2j , …, w mj}, wherein w mj represents the weight of the mth performance indicator in the node j, and a larger value indicates that the performance is better. The task can also be divided into different types, and each type is assigned a different weight to represent its demand for different node performances. Thus, the adaptability of each node to the task can be better evaluated.

[0024] According to an embodiment of the present application, the method further comprises: completing the calculation of the task by the nodes in the set and outputting the calculation result. After outputting the set, the task can be sent to each node in the set to complete the calculation and return the result to the user.

[0025] The computing power scheduling system according to the present application is configured to perform the computing power scheduling method as described above, and comprises a task receiving module configured to perform steps S1 and S2, and a node selection module configured to perform steps S3 to S5.

[0026] According to an embodiment of the present application, the node selection module is further configured to send the task to the nodes in the set for completion of the computation by the nodes.

[0027] According to an embodiment of the present application, the computing power scheduling system further comprises a task returning module configured to receive the computation result from the nodes that complete the computation and return the computation result to the task sender.

[0028] In another aspect, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores computer instructions, and the computer instructions are executed by a processor to implement the computing power scheduling method as described above.

[0029] Obviously, the design schemes as described above for the computing power scheduling method, the computing power scheduling system and the computer readable storage medium can be used together and combined with each other. The advantages of the computing power scheduling method as described above are correspondingly applicable to the computing power scheduling system and the computer readable storage medium. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings are used to provide a further understanding of the technical schemes of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the technical schemes of the present application, and do not constitute a limitation on the technical schemes of the present application.

[0031] Figure 1 A flow chart of an embodiment of the computing power scheduling method of the present application is shown,

[0032] Figure 2 A schematic diagram of an embodiment of the computing power scheduling system of the present application is shown. DETAILED DESCRIPTION

[0033] In order to make the objects, technical schemes and advantages of the embodiments of the present application more clear, the specific embodiments of the present application are described in detail below with reference to the drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0034] Figure 1 A flow chart of an embodiment of the computing power scheduling method of the present application is shown. The computing power scheduling method according to the present application comprises:

[0035] Step S1: receiving a task, wherein the task contains a requirement for node performance;

[0036] Step S2: determining a current available node set, the node set including a new node set and an existing node set, wherein the nodes in the new node set accept tasks less than a certain threshold, and the nodes in the existing node set accept tasks greater than or equal to the certain threshold;

[0037] Step S3: randomly selecting m nodes from the new node set, and adding the m nodes to the set;

[0038] Step S4: selecting q t T p u nodes with the largest values from the existing node set, and adding the n nodes to the set:

[0039]

[0040] wherein q * and p * are factor vectors, u is a single node, t is a single task, K is a training set, r ut is a label result (0 or 1) of a (u, t) sample in the training set, wherein 1 represents that the previous task t is assigned to the node u, and 0 represents that the node u is not assigned, q t is a vector associated with the task t generated by q * , p u is a vector associated with the node u generated by p * , q t T p u represents a predicted value of the node u and the task t;

[0041] Step S5: outputting the set.

[0042] In order to better evaluate the adaptability of each node to the task, the task can be divided into different types, and each type is assigned a different weight to represent the demand of the node for different performance. In addition, the performance of each node can be represented by different weights, and the greater the value, the better the performance.

[0043] In the embodiment, the nodes in the node set respectively have performance information, and the performance information includes the following 7 indexes to evaluate the performance of the nodes from different dimensions:

[0044] (1) Node bandwidth, indicating the bandwidth level of the node. For each node, the objective performance such as bandwidth, CPU, I / O read-write capability, etc. is recorded first, and each index is divided into, for example, 5 levels according to the performance.

[0045] (2) Node computing resource, representing the computing resource level of the node, since there is a certain correlation between CPU, SSD and processor, it is uniformly represented by computing resource, similar to node bandwidth, and the various indicators are divided into, for example, 5 levels according to the performance.

[0046] (3) Node activity, which mainly records the online duration of the node. The longer the online duration, the higher the activity, and the probability of obtaining the task increases.

[0047] (4) The number of accepted tasks, which records the number of accepted tasks of a certain type. In order to reward and distribute points, it is also possible to better balance the distribution of tasks and allow more nodes to participate in the system. The number of accepted tasks, the number of rewarded times and other indicators are set.

[0048] (5) The number of rewarded times, which records the number of rewarded times for completing a certain type of task.

[0049] (6) Average completion time, indicating the average completion time of the node for the task. By recording the completion of the node executing the task, the adaptability of the node to different tasks can be more accurately identified.

[0050] (7) Delay time, which is updated before each task. The network delay of each node is tested before the task is distributed, so as to determine the current network state of the node.

[0051] After receiving the task, the current available node set is first divided into two parts, including the newly added node set and the existing node set. For the newly added nodes, in order to ensure that each node can be selected fairly, the idea of randomly selecting these nodes in each task allocation process is proposed, thereby forming a random scheduling strategy. For other existing nodes that have been running and providing services, the first n nodes with the highest efficiency are selected by solving formula (1).

[0052] In step S4, the purpose of the algorithm is to learn the factor vectors q * and p * . In formula (1), for a given task t, q t represents the vector associated with task t generated by q * , and for a given node u, p u represents the user preference feature generated by p * , through q t T p u obtains the predicted value between node u and task t.

[0053] In this embodiment, formula (1) is solved using a gradient descent algorithm. The related prediction error e ut , q t and pu The following update can be performed until convergence.

[0054]

[0055] q t ←q t +γ×(e ut ×p u -λ×q t ) (3)

[0056] p u ←p u +γ×(e ut ×q t -λ×p u ) (4)

[0057] where the parameter λ is used to control the degree of regularization, representing the learning rate in the gradient algorithm.

[0058] The meanings of the parameter symbols are shown in Table 1 below:

[0059]

[0060] The specific steps of the computing power scheduling method are shown in Table 2 below. The purpose of the computing power scheduling method is to first select m new nodes from the new node set that have less than a certain threshold number of accepted tasks, and then select n nodes from the existing node set based on the training set K. In the algorithm process, the recommended node set Φ is first initialized to empty (line 1 of the algorithm). Lines 2-5 represent randomly selecting m new nodes from the new node set , and lines 6-11 learn the training set K. In this process, e ut , q t and p u will be updated according to the gradient descent algorithm until they meet the requirements of formula (2), at which point the loop is stopped and the algorithm selects the first n nodes with the largest rate in q t T p u (line 12). Finally, the algorithm selects a set of nodes Φ and returns.

[0061] Table 2 Specific steps of the computing power scheduling method

[0062]

[0063] Figure 2 A schematic diagram of one embodiment of the computing power scheduling system of the present application is shown.

[0064] The computing power scheduling system according to the present application is configured to perform the computing power scheduling method. The computing power scheduling system comprises a task receiving module configured to perform steps S1 and S2, i.e. receiving a task and determining a current available node set (scheduling nodes). The computing power scheduling system further comprises a node selection module configured to perform steps S3 to S5. The node selection module randomly selects m nodes from the new node set according to steps S3 to S5, and selects n nodes from the existing node set by solving the formula, thus a total of m+n nodes are selected for the received task. In this embodiment, the node selection module is further configured to send the task to the nodes in the recommended node set, so that the m+n nodes complete the computation of the task.

[0065] In this embodiment, the computing power scheduling system further comprises a task returning module configured to receive the computation result from the nodes that complete the computation and return the computation result to the task sender, i.e. the user.

[0066] The embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict. Those skilled in the art will understand that various changes in form and details can be made without departing from the scope of the present disclosure set forth by the appended claims.

Claims

1. A computing power scheduling method, comprising: Step S1: Receive the task, which includes the requirements for node performance; Step S2: Determine the currently available set of nodes, which includes a new set of nodes and an existing set of nodes, wherein the number of times the nodes in the new set of nodes accept tasks is less than a specific threshold, and the number of times the nodes in the existing set of nodes accept tasks is greater than or equal to a specific threshold. Step S3: Randomly select m nodes from the new node set and add the m nodes to the set; Step S4: Select q from the existing node set by solving the following formula. t T p u The n nodes with the largest values ​​are added to the set: Where, q * and p * It is a factor vector, where u is a single node, t is a single task, K is the training set, and r is a factor vector. ut It represents the labeling results (0 or 1) of samples (u,t) in the training set, where 1 indicates that the previous task t was assigned to node u, and 0 indicates that it was not assigned. t Through q * The generated vector p associated with task t u Through p * The generated vector q associated with node u t T p u This represents the predicted values ​​for node u and task t, with parameter λ controlling the degree of regularization. Step S5: Output the set, where the nodes in the set complete the task calculation and output the calculation result.

2. The computing power scheduling method according to claim 1, characterized in that, Each node in the node set has performance information, which includes at least one of the following metrics: bandwidth, computing resources, activity level, number of tasks accepted, number of rewards, average completion time, and latency.

3. The computing power scheduling method according to claim 1, characterized in that, The received task contains the values ​​of m and n.

4. The computing power scheduling method according to claim 3, characterized in that, The formula is solved using the gradient descent algorithm, based on the training set K and the associated prediction error e. ut q t and p u Perform the following updates until convergence: q t ←q t +γ×(e ut ×p u -λ×q t ) p u ←p u +γ×(e ut ×q t -λ×p u ) Here, the parameter λ is used to control the degree of regularization, and γ represents the learning rate in the gradient algorithm.

5. The computing power scheduling method according to claim 1, characterized in that, The performance of each node is represented as N. j ={w 1j ,w 2j ,…,w mj }, where w mj This represents the weight of the m-th performance metric in node j; a larger value indicates better performance for that metric.

6. A computing power scheduling system, the computing power scheduling system being used to execute the computing power scheduling method according to any one of claims 1-5, the computing power scheduling system comprising: The task receiving module is configured to execute steps S1 and S2; and The node selection module is configured to perform steps S3 to S5.

7. The computing power scheduling system according to claim 6, characterized in that, The node selection module is also configured to send tasks to nodes in the set so that these nodes can perform the task computation.

8. The computing power scheduling system according to claim 7, characterized in that, The computing power scheduling system also includes a task return module, which is configured to receive the calculation results from the node that has completed the calculation and return the calculation results to the task sender.

9. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method according to any one of claims 1 to 5.

Images