Task scheduling method, device, electronic device and storage medium

Abstract

The present invention relates to the field of deep learning technology, and discloses a task scheduling method, comprising: obtaining tasks to be scheduled on a scheduling platform; generating a pairwise comparison matrix of the tasks to be scheduled based on multiple scheduling indicators; calculating the eigenvalues ​​and eigenvectors of the pairwise comparison matrix; determining the weights of the multiple scheduling indicators based on the eigenvalues ​​and eigenvectors; obtaining multiple scheduling schemes for the tasks to be scheduled, and calculating the scheme scores of the multiple scheduling schemes based on the weights of the multiple scheduling indicators; determining the scheduling scheme with the highest scheme score as the scheme to be scheduled; and scheduling the tasks to be scheduled according to the scheme to be scheduled. At the same time, the present invention also proposes a task scheduling device and a computer-readable storage medium. In addition, the present invention also relates to blockchain technology, and the scheduling scheme can be stored in a blockchain node. The present invention can achieve efficient task scheduling.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a task scheduling method, device, electronic device, and computer-readable storage medium. Background Art

[0002] In the field of deep learning, deep learning algorithms are typically run and researched by scheduling deep learning tasks within a deep learning system. Inefficient scheduling prevents deep learning tasks from completing quickly and also wastes computing resources. Therefore, a method for efficient task scheduling is urgently needed. Summary of the Invention

[0003] The present invention provides a task scheduling method, device, electronic device and computer-readable storage medium, the main purpose of which is to provide an efficient task scheduling method.

[0004] To achieve the above object, the present invention provides a task scheduling method, comprising:

[0005] Get the tasks to be scheduled on the scheduling platform;

[0006] generating a pairwise comparison matrix of the tasks to be scheduled according to a plurality of scheduling indicators, wherein the scheduling indicators include indicators related to system resources of the scheduling platform and / or indicators related to the tasks to be scheduled;

[0007] Calculating the eigenvalues ​​and eigenvectors of the pairwise comparison matrix;

[0008] Determining weights of the plurality of scheduling indicators according to the eigenvalues ​​and eigenvectors;

[0009] Obtaining multiple scheduling schemes for the task to be scheduled, and calculating scheme scores of the multiple scheduling schemes according to weights of the multiple scheduling indicators;

[0010] Determine the scheduling plan with the highest plan score as the plan to be scheduled;

[0011] The tasks to be scheduled are scheduled according to the scheduling plan.

[0012] Optionally, generating the pairwise comparison matrix of the tasks to be scheduled according to the multiple scheduling indicators includes:

[0013] Determining the priorities of the plurality of scheduling indicators according to the attribute information of the task to be scheduled;

[0014] Initializing the values ​​of designated elements in the pairwise comparison matrix according to the priority and the priority-value correspondence table;

[0015] The values ​​of other elements in the pairwise comparison matrix other than the designated element are determined according to the consistency condition.

[0016] Optionally, calculating the scheme scores of the multiple scheduling schemes according to the weights of the multiple scheduling indicators includes:

[0017] Calculating a sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to a different scheduling indicator in the multiple scheduling indicators;

[0018] The calculated scores of the multiple sub-scheduling schemes of each scheduling scheme are weighted according to the weights of the multiple scheduling indicators to obtain scheme scores of the multiple scheduling schemes.

[0019] Optionally, the different scheduling indicators include GPU concentration, CPU / memory balance, and communication indicators, and calculating the sub-scheduling scheme scores of each scheduling scheme in the multiple scheduling schemes corresponding to different scheduling indicators in the multiple scheduling indicators includes:

[0020] Calculating a sub-scheduling scheme score corresponding to the GPU concentration for each scheduling scheme in the multiple scheduling schemes according to the remaining number of GPUs in the scheduling platform;

[0021] Calculating a sub-scheduling scheme score corresponding to the CPU / memory balance degree for each scheduling scheme in the multiple scheduling schemes according to the remaining number of CPUs and memory amounts of the scheduling platform;

[0022] A sub-scheduling scheme score corresponding to the communication indicator of each scheduling scheme in the multiple scheduling schemes is calculated according to the scheduling task volume of the scheduling platform.

[0023] Optionally, the calculating the sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the GPU concentration according to the remaining number of GPUs of the scheduling platform includes: determining the required number of GPUs of the task to be scheduled;

[0024] Obtain a node set in the scheduling platform whose number of remaining GPUs is greater than or equal to the required number of GPUs;

[0025] Determine a remaining minimum value and a remaining maximum value of the number of remaining GPUs in the node set;

[0026] A sub-scheduling scheme score corresponding to the GPU concentration of each scheduling scheme in the multiple scheduling schemes is calculated according to the remaining minimum value and the remaining maximum value.

[0027] Optionally, the calculating, based on the remaining number of CPUs and memory amounts of the scheduling platform, a sub-scheduling scheme score corresponding to the CPU / memory balance degree for each of the multiple scheduling schemes includes:

[0028] Obtain the remaining number of CPUs and memory of each node in the scheduling platform, and generate an idle matrix based on the remaining number of CPUs and memory;

[0029] Determining a CPU and memory requirement vector of the task to be scheduled based on the attribute information of the task to be scheduled;

[0030] A sub-scheduling scheme score corresponding to the CPU / memory balance degree of each scheduling scheme in the multiple scheduling schemes is calculated according to the demand vector and the idle matrix.

[0031] Optionally, the calculating, according to the scheduling task volume of the scheduling platform, a sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the communication indicator includes:

[0032] Obtaining other scheduling tasks in each node in the scheduling platform that have a communication dependency relationship with the task to be scheduled, and obtaining multiple subtask quantities;

[0033] Counting the amount of the multiple subtasks to obtain the total amount of tasks;

[0034] A sub-scheduling scheme score corresponding to the communication indicator of each scheduling scheme in the multiple scheduling schemes is calculated according to the total task amount and the multiple sub-task amounts.

[0035] In order to solve the above problems, the present invention further provides a task scheduling device, which includes:

[0036] The task acquisition module is used to obtain the tasks to be scheduled on the scheduling platform;

[0037] a matrix generation module, configured to generate a pairwise comparison matrix of the tasks to be scheduled based on a plurality of scheduling indicators, wherein the scheduling indicators include indicators related to system resources of the scheduling platform and / or indicators related to the tasks to be scheduled;

[0038] A feature calculation module, used to calculate the eigenvalues ​​and eigenvectors of the pairwise comparison matrix;

[0039] A weight determination module, configured to determine the weights of the plurality of scheduling indicators according to the eigenvalues ​​and eigenvectors;

[0040] A score calculation module is used to obtain multiple scheduling schemes for the task to be scheduled, and calculate the scheme scores of the multiple scheduling schemes according to the weights of the multiple scheduling indicators;

[0041] A scheme determination module is used to determine the scheduling scheme with the highest scheme score as the scheme to be scheduled;

[0042] The scheduling module is used to schedule the tasks to be scheduled according to the scheduling plan.

[0043] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0044] a memory storing at least one computer program instruction; and

[0045] The processor executes the computer program instructions stored in the memory to implement the task scheduling method described above.

[0046] In order to solve the above problem, the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the above-mentioned task scheduling method.

[0047] When scheduling tasks, the embodiments of the present invention calculate the solution scores of different scheduling solutions based on multiple scheduling indicators, and then determine the undetermined angle solution. This avoids problems such as the inability to execute scheduled tasks or the large resource consumption of scheduled tasks that affects the execution of other scheduled tasks when scheduling based on a single scheduling indicator. This improves the reliability and effectiveness of scheduling, thereby increasing scheduling efficiency. Therefore, the task scheduling method, device, and computer-readable storage medium proposed in the present invention can achieve the purpose of efficient scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a task scheduling method provided by one embodiment of the present invention;

[0049] Figure 2 A module diagram of a task scheduling device provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a task scheduling method provided by an embodiment of the present invention;

[0051] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0053] The present application provides a task scheduling method. The task scheduling method may be performed by at least one electronic device, such as a server or a terminal, that can be configured to perform the method provided by the present application. In other words, the task scheduling method may be performed by software or hardware installed on a terminal or server device, where the software may be a blockchain platform. The server may include, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0054] Reference Figure 1 FIG. 1 is a flow chart of a task scheduling method according to an embodiment of the present invention. In this embodiment, the task scheduling method includes:

[0055] S1. Obtain the tasks to be scheduled on the scheduling platform.

[0056] In this embodiment, the scheduling platform is a lightweight distributed task scheduling platform, which can receive scheduling requests to generate tasks to be scheduled, and can also process the scheduling requests according to a scheduling plan.

[0057] The tasks to be scheduled are various events for which resources are allocated by a computer for computational processing. Specifically, the tasks to be scheduled in the embodiment of the present invention are computational tasks for deep learning.

[0058] S2. Generate a pairwise comparison matrix of the tasks to be scheduled according to a plurality of scheduling indicators, where the scheduling indicators include indicators related to system resources of the scheduling platform and / or indicators related to the tasks to be scheduled.

[0059] In this embodiment, the scheduling indicator is a factor that can affect scheduling allocation.

[0060] Preferably, in an embodiment of the present invention, the scheduling indicators include GPU concentration, CPU / memory balance and communication indicators.

[0061] Optionally, in other embodiments of the present invention, the scheduling indicators include but are not limited to network structure, antenna technology, and uplink power.

[0062] During specific implementation, the tasks to be scheduled often require various computer resources, such as communication for data transmission, CPU and memory for data calculation, and GPU resources for image processing. However, when the main tasks in each task to be scheduled are different, the degree of demand for various resources is also different. Therefore, integrating multiple scheduling indicators and selecting the optimal scheduling plan can improve scheduling efficiency and ensure that the scheduled tasks run efficiently and smoothly.

[0063] In detail, the number of row elements and column elements of the pairwise comparison matrix is ​​the same, and the values ​​of the row elements are correlated with the values ​​of the column elements.

[0064] In detail, generating the pairwise comparison matrix of the tasks to be scheduled according to the multiple scheduling indicators includes:

[0065] Determining the priorities of the plurality of scheduling indicators according to the attribute information of the task to be scheduled;

[0066] Initializing the values ​​of designated elements in the pairwise comparison matrix according to the priority and the priority-value correspondence table;

[0067] The values ​​of other elements in the pairwise comparison matrix other than the designated element are determined according to the consistency condition.

[0068] Furthermore, the pairwise comparison matrix includes:

[0069]

[0070] Among them, A is the pairwise comparison matrix, a ij Indicates the importance of the i-th scheduling indicator relative to the j-th scheduling indicator, and the pairwise comparison matrix is ​​a positive reciprocal matrix that satisfies the consistency condition, which is a ij *a ji =1 and

[0071] Furthermore, the priority and value correspondence table described in the embodiment of the present invention is a value table constructed based on a nine-level scaling method. The priority and value correspondence table divides the priority into 9 levels and corresponds to values ​​1-9. The higher the priority, the higher the value.

[0072] For example, in deep learning task scheduling, since the demand for CPU or memory resources is higher than the shortage of GPU resources, the priority of CPU / memory balance is higher than GPU concentration. 12 =5; if the task to be scheduled has no communication requirements, indicating that the communication indicator has the lowest priority, and the GPU concentration has a higher priority, set a according to the priority and value correspondence table 13 =9.

[0073] S3. Calculate the eigenvalues ​​and eigenvectors of the pairwise comparison matrix.

[0074] In detail, the embodiment of the present invention calculates the eigenvalues ​​and eigenvectors of the pairwise comparison matrix through the equation Ax=λx.

[0075] Wherein, A is the pairwise comparison matrix, λ is a numerical value, and x is a non-zero vector. If there exists a number λ and a non-zero vector x such that the equation Ax=λx holds, then the number λ is the eigenvalue of the matrix A, and the non-zero vector x is the eigenvector of the corresponding eigenvalue λ of the matrix A.

[0076] By solving the equation, the embodiment of the present invention can obtain multiple eigenvalues ​​of the pairwise comparison matrix and multiple eigenvectors corresponding to the multiple eigenvalues.

[0077] S4. Determine weights of the multiple scheduling indicators according to the eigenvalues ​​and eigenvectors.

[0078] In detail, the eigenvalue includes multiple eigenvalues, and determining the weights of the multiple scheduling indicators based on the eigenvalues ​​and eigenvectors includes:

[0079] Selecting a maximum eigenvalue from the plurality of eigenvalues;

[0080] The eigenvector corresponding to the maximum eigenvalue is normalized to obtain weights of the multiple scheduling indicators.

[0081] In this embodiment, linear function conversion, logarithmic function conversion, and inverse cotangent function conversion may be used for normalization.

[0082] In the embodiment of the present invention, the eigenvector corresponding to the maximum eigenvalue is normalized and recorded as W=(w1, w2, w3), where w1, w2, and w3 are specific weight values ​​corresponding to each scheduling indicator.

[0083] The embodiment of the present invention can limit the feature vector to a certain range through normalization processing, which is conducive to comparison and weighting of data of different units or magnitudes, thereby improving the efficiency of data processing.

[0084] S5. Acquire multiple scheduling schemes for the tasks to be scheduled, and calculate scheme scores of the multiple scheduling schemes according to weights of the multiple scheduling indicators.

[0085] In the embodiment of the present invention, the multiple scheduling schemes may be multiple scheduling schemes determined according to preset scheduling conditions in the scheduling platform, or the multiple scheduling schemes may also be preset scheduling schemes.

[0086] Preferably, the scheduling plan can also be obtained from a node of a blockchain.

[0087] By storing the scheduling plan in the blockchain, the security of the scheduling plan can be further improved.

[0088] In detail, the calculating of the scheme scores of the multiple scheduling schemes according to the weights of the multiple scheduling indicators includes:

[0089] Calculating a sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to a different scheduling indicator in the multiple scheduling indicators;

[0090] The calculated scores of the multiple sub-scheduling schemes of each scheduling scheme are weighted according to the weights of the multiple scheduling indicators to obtain scheme scores of the multiple scheduling schemes.

[0091] For example, the scores of GPU concentration, CPU / memory balance and communication index for a scheduling scheme are 1, 1 / 2 and 3 / 5 respectively, and the weights of GPU concentration, CPU / memory balance and communication index are 2 / 3, 1 / 6 and 1 / 6 respectively. Then the weight calculation is performed to obtain The scheduling plan score is 51 / 60.

[0092] Furthermore, when the different scheduling indicators include GPU concentration, CPU / memory balance, and communication indicators, calculating the sub-scheduling scheme scores of each scheduling scheme in the multiple scheduling schemes corresponding to different scheduling indicators in the multiple scheduling indicators includes:

[0093] Calculating a sub-scheduling scheme score corresponding to the GPU concentration for each scheduling scheme in the multiple scheduling schemes according to the remaining number of GPUs in the scheduling platform;

[0094] Calculating a sub-scheduling scheme score corresponding to the CPU / memory balance degree for each scheduling scheme in the multiple scheduling schemes according to the remaining number of CPUs and memory amounts of the scheduling platform;

[0095] A sub-scheduling scheme score corresponding to the communication indicator of each scheduling scheme in the multiple scheduling schemes is calculated according to the scheduling task volume of the scheduling platform.

[0096] In detail, the calculating, according to the number of remaining GPUs of the scheduling platform, the sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the GPU concentration includes:

[0097] Determine the number of GPUs required for the task to be scheduled;

[0098] Obtain a node set in the scheduling platform whose number of remaining GPUs is greater than or equal to the required number of GPUs;

[0099] Determine a remaining minimum value and a remaining maximum value of the number of remaining GPUs in the node set;

[0100] A sub-scheduling scheme score corresponding to the GPU concentration of each scheduling scheme in the multiple scheduling schemes is calculated according to the remaining minimum value and the remaining maximum value.

[0101] Specifically, a specific method of calculating the sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the GPU concentration according to the remaining minimum value and the remaining maximum value is as follows:

[0102]

[0103] Among them, s k1 Represents the scheduling plan P kThe sub-scheduling solution score corresponding to the GPU concentration, f gk represents the number of remaining GPUs on node k, r g Indicates the number of GPUs required for the task to be scheduled, Indicates the remaining minimum value among the remaining GPU numbers, Indicates the maximum number of remaining GPUs.

[0104] In this embodiment, by calculating the sub-scheduling scheme scores corresponding to the GPU concentration of each scheduling scheme, the problem of sufficient idle GPUs in total but insufficient idle GPUs in each node can be avoided, thereby preventing the scheduling task from being unable to run and improving the scheduling efficiency.

[0105] Specifically, the calculating of the sub-scheduling scheme score of each scheduling scheme corresponding to the CPU / memory balance degree in the multiple scheduling schemes according to the remaining number of CPUs and memory amounts of the scheduling platform includes:

[0106] Obtain the remaining number of CPUs and memory of each node in the scheduling platform, and generate an idle matrix based on the remaining number of CPUs and memory;

[0107] Determining a CPU and memory requirement vector of the task to be scheduled based on the attribute information of the task to be scheduled;

[0108] A sub-scheduling scheme score corresponding to the CPU / memory balance degree of each scheduling scheme in the multiple scheduling schemes is calculated according to the demand vector and the idle matrix.

[0109] Specifically, the specific method of calculating the sub-scheduling scheme score of each scheduling scheme corresponding to the CPU / memory balance degree in the multiple scheduling schemes according to the demand vector and the idle matrix is ​​as follows:

[0110]

[0111] Among them, s k2 The scheduling plan P k The sub-scheduling scheme score corresponding to the CPU / memory balance, (r c ,r m ) is the demand vector of the task to be scheduled for CPU and memory, (f ck ,f mk ) is the idle matrix of CPU and memory on node k.

[0112] In this embodiment, the problem of load imbalance in the scheduling process can be solved by calculating the sub-scheduling scheme scores corresponding to the CPU / memory balance of each scheduling scheme.

[0113] In detail, the calculating, based on the scheduling task volume of the scheduling platform, a sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the communication indicator includes:

[0114] Obtaining other scheduling tasks in each node in the scheduling platform that have a communication dependency relationship with the task to be scheduled, and obtaining multiple subtask quantities;

[0115] Counting the amount of the multiple subtasks to obtain the total amount of tasks;

[0116] A sub-scheduling scheme score corresponding to the communication indicator of each scheduling scheme in the multiple scheduling schemes is calculated according to the total task amount and the multiple sub-task amounts.

[0117] Specifically, the specific method of calculating the sub-scheduling scheme score of each scheduling scheme corresponding to the communication indicator in the multiple scheduling schemes according to the total task amount and the multiple sub-task amounts is as follows:

[0118]

[0119] Among them, s k3 is the scheduling plan P k The sub-scheduling scheme score corresponding to the communication indicator, n is the total number of tasks, n k is the number of subtasks located on node k.

[0120] In this embodiment, by calculating the sub-scheduling scheme scores of the communication indicators corresponding to each scheduling scheme, scheduling failures caused by network communication delays can be reduced, the success rate and reliability of scheduling can be improved, and the efficiency of scheduling can be improved.

[0121] S6. Determine the scheduling plan with the highest plan score as the plan to be scheduled.

[0122] In the embodiment of the present invention, the scheme scores of the various scheduling schemes are compared to determine the scheduling scheme corresponding to the highest scheme score as the scheme to be scheduled.

[0123] S7. Schedule the tasks to be scheduled according to the scheduling plan.

[0124] After determining the to-be-scheduled scheme, the embodiment of the present invention schedules the to-be-scheduled tasks according to the to-be-scheduled scheme.

[0125] When performing task scheduling, resources within the computer are allocated to the tasks to be scheduled according to the scheduling plan so that the tasks to be scheduled can be executed and completed, thereby ensuring the ultimate implementation of the deep learning algorithm on the computer.

[0126] When scheduling tasks, the embodiments of the present invention calculate the solution scores of different scheduling solutions based on multiple scheduling indicators, and then determine the undetermined angle solution. This avoids problems such as the inability to execute scheduled tasks or the large resource consumption of scheduled tasks that affects the execution of other scheduled tasks when scheduling based on a single scheduling indicator. This improves the reliability and effectiveness of scheduling, thereby increasing scheduling efficiency. Therefore, the task scheduling method proposed in the present invention can achieve the goal of efficient scheduling.

[0127] like Figure 2 FIG. 1 is a functional module diagram of the task scheduling device of the present invention.

[0128] The task scheduling device 100 described in the present invention can be installed in an electronic device. Depending on the functionality implemented, the task scheduling device may include a task acquisition module 101, a matrix generation module 102, a feature calculation module 103, a weight determination module 104, a score calculation module 105, a solution determination module 106, and a scheduling module 107. A module, also referred to as a unit, is a series of computer program segments that can be executed by an electronic device processor and perform a fixed function, and is stored in the memory of the electronic device.

[0129] In this embodiment, the functions of each module / unit are as follows:

[0130] The task acquisition module 101 is used to acquire tasks to be scheduled on the scheduling platform.

[0131] In this embodiment, the scheduling platform is a lightweight distributed task scheduling platform, which can receive scheduling requests to generate tasks to be scheduled, and can also process the scheduling requests according to a scheduling plan.

[0132] The tasks to be scheduled are various events for which resources are allocated by a computer for computational processing. Specifically, the tasks to be scheduled in the embodiment of the present invention are computational tasks for deep learning.

[0133] The matrix generation module 102 is configured to generate a pairwise comparison matrix of the tasks to be scheduled based on a plurality of scheduling indicators, wherein the scheduling indicators include indicators related to the system resources of the scheduling platform and / or indicators related to the tasks to be scheduled.

[0134] In this embodiment, the scheduling indicator is a factor that can affect scheduling allocation.

[0135] Preferably, in an embodiment of the present invention, the scheduling indicators include GPU concentration, CPU / memory balance and communication indicators.

[0136] Optionally, in other embodiments of the present invention, the scheduling indicators include but are not limited to network structure, antenna technology, and uplink power.

[0137] During specific implementation, the tasks to be scheduled often require various computer resources, such as communication for data transmission, CPU and memory for data calculation, and GPU resources for image processing. However, when the main tasks in each task to be scheduled are different, the degree of demand for various resources is also different. Therefore, integrating multiple scheduling indicators and selecting the optimal scheduling plan can improve scheduling efficiency and ensure that the scheduled tasks run efficiently and smoothly.

[0138] In detail, the number of row elements and column elements of the pairwise comparison matrix is ​​the same, and the values ​​of the row elements are correlated with the values ​​of the column elements.

[0139] In detail, the matrix generation module 102 is specifically used for:

[0140] Determining the priorities of the plurality of scheduling indicators according to the attribute information of the task to be scheduled;

[0141] Initializing the values ​​of designated elements in the pairwise comparison matrix according to the priority and the priority-value correspondence table;

[0142] The values ​​of other elements in the pairwise comparison matrix other than the designated element are determined according to the consistency condition.

[0143] Furthermore, the pairwise comparison matrix includes:

[0144]

[0145] Among them, A is the pairwise comparison matrix, a ij Indicates the importance of the i-th scheduling indicator relative to the j-th scheduling indicator, and the pairwise comparison matrix is ​​a positive reciprocal matrix that satisfies the consistency condition, which is a ij *a ji =1 and

[0146] Furthermore, the priority and value correspondence table described in the embodiment of the present invention is a value table constructed based on a nine-level scaling method. The priority and value correspondence table divides the priority into 9 levels and corresponds to values ​​1-9. The higher the priority, the higher the value.

[0147] For example, in deep learning task scheduling, since the demand for CPU or memory resources is higher than the shortage of GPU resources, the priority of CPU / memory balance is higher than GPU concentration. 12 =5; if the task to be scheduled has no communication requirements, indicating that the communication indicator has the lowest priority, and the GPU concentration has a higher priority, set a according to the priority and value correspondence table13 =9.

[0148] The feature calculation module 103 is used to calculate the eigenvalues ​​and eigenvectors of the pairwise comparison matrix.

[0149] In detail, the embodiment of the present invention calculates the eigenvalues ​​and eigenvectors of the pairwise comparison matrix through the equation Ax=λx.

[0150] Wherein, A is the pairwise comparison matrix, λ is a numerical value, and x is a non-zero vector. If there exists a number λ and a non-zero vector x such that the equation Ax=λx holds, then the number λ is the eigenvalue of the matrix A, and the non-zero vector x is the eigenvector of the corresponding eigenvalue λ of the matrix A.

[0151] By solving the equation, the embodiment of the present invention can obtain multiple eigenvalues ​​of the pairwise comparison matrix and multiple eigenvectors corresponding to the multiple eigenvalues.

[0152] The weight determination module 104 is configured to determine the weights of the plurality of scheduling indicators according to the eigenvalues ​​and eigenvectors.

[0153] In detail, the eigenvalue includes multiple eigenvalues, and the weight determination module 104 is specifically used to:

[0154] Selecting a maximum eigenvalue from the plurality of eigenvalues;

[0155] The eigenvector corresponding to the maximum eigenvalue is normalized to obtain weights of the multiple scheduling indicators.

[0156] In this embodiment, linear function conversion, logarithmic function conversion, and inverse cotangent function conversion may be used for normalization.

[0157] In the embodiment of the present invention, the eigenvector corresponding to the maximum eigenvalue is normalized and recorded as W=(w1, w2, w3), where w1, w2, and w3 are specific weight values ​​corresponding to each scheduling indicator.

[0158] The embodiment of the present invention can limit the feature vector to a certain range through normalization processing, which is conducive to comparison and weighting of data of different units or magnitudes, thereby improving the efficiency of data processing.

[0159] The score calculation module 105 obtains multiple scheduling solutions for the task to be scheduled, and calculates solution scores of the multiple scheduling solutions according to the weights of the multiple scheduling indicators.

[0160] In the embodiment of the present invention, the multiple scheduling schemes may be multiple scheduling schemes determined according to preset scheduling conditions in the scheduling platform, or the multiple scheduling schemes may also be preset scheduling schemes.

[0161] Preferably, the scheduling plan can also be obtained from a node of a blockchain.

[0162] By storing the scheduling plan in the blockchain, the security of the scheduling plan can be further improved.

[0163] In detail, the score calculation module 105 is specifically used to:

[0164] An acquiring unit, configured to acquire a plurality of scheduling schemes for the task to be scheduled;

[0165] a sub-scheduling scheme score calculation unit, configured to calculate a sub-scheduling scheme score of each of the plurality of scheduling schemes corresponding to a different scheduling indicator in the plurality of scheduling indicators;

[0166] The weight calculation unit is used to perform weight calculation on the calculated scores of the multiple sub-scheduling schemes of each scheduling scheme according to the weights of the multiple scheduling indicators to obtain scheme scores of the multiple scheduling schemes.

[0167] For example, the scores of GPU concentration, CPU / memory balance and communication index for a scheduling scheme are 1, 1 / 2 and 3 / 5 respectively, and the weights of GPU concentration, CPU / memory balance and communication index are 2 / 3, 1 / 6 and 1 / 6 respectively. Then the weight calculation is performed to obtain The scheduling plan score is 51 / 60.

[0168] Furthermore, when the different scheduling indicators include GPU concentration, CPU / memory balance, and communication indicators, the sub-scheduling scheme score calculation unit is specifically used to:

[0169] Calculating a sub-scheduling scheme score corresponding to the GPU concentration for each scheduling scheme in the multiple scheduling schemes according to the remaining number of GPUs in the scheduling platform;

[0170] Calculating a sub-scheduling scheme score corresponding to the CPU / memory balance degree for each scheduling scheme in the multiple scheduling schemes according to the remaining number of CPUs and memory amounts of the scheduling platform;

[0171] A sub-scheduling scheme score corresponding to the communication indicator of each scheduling scheme in the multiple scheduling schemes is calculated according to the scheduling task volume of the scheduling platform.

[0172] In detail, the calculating, according to the number of remaining GPUs of the scheduling platform, the sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the GPU concentration includes:

[0173] Determine the number of GPUs required for the task to be scheduled;

[0174] Obtain a node set in the scheduling platform whose number of remaining GPUs is greater than or equal to the required number of GPUs;

[0175] Determine a remaining minimum value and a remaining maximum value of the number of remaining GPUs in the node set;

[0176] A sub-scheduling scheme score corresponding to the GPU concentration of each scheduling scheme in the multiple scheduling schemes is calculated according to the remaining minimum value and the remaining maximum value.

[0177] Specifically, a specific method of calculating the sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the GPU concentration according to the remaining minimum value and the remaining maximum value is as follows:

[0178]

[0179] Among them, s k1 Represents the scheduling plan P k The sub-scheduling solution score corresponding to the GPU concentration, f gk represents the number of remaining GPUs on node k, r g Indicates the number of GPUs required for the task to be scheduled, Indicates the remaining minimum value among the remaining GPU numbers, Indicates the maximum number of remaining GPUs.

[0180] In this embodiment, by calculating the sub-scheduling scheme scores corresponding to the GPU concentration of each scheduling scheme, the problem of sufficient idle GPUs in total but insufficient idle GPUs in each node can be avoided, thereby preventing the scheduling task from being unable to run and improving the scheduling efficiency.

[0181] Specifically, the calculating of the sub-scheduling scheme score of each scheduling scheme corresponding to the CPU / memory balance degree in the multiple scheduling schemes according to the remaining number of CPUs and memory amounts of the scheduling platform includes:

[0182] Obtain the remaining number of CPUs and memory of each node in the scheduling platform, and generate an idle matrix based on the remaining number of CPUs and memory;

[0183] Determining a CPU and memory requirement vector of the task to be scheduled based on the attribute information of the task to be scheduled;

[0184] A sub-scheduling scheme score corresponding to the CPU / memory balance degree of each scheduling scheme in the multiple scheduling schemes is calculated according to the demand vector and the idle matrix.

[0185] Specifically, the specific method of calculating the sub-scheduling scheme score of each scheduling scheme corresponding to the CPU / memory balance degree in the multiple scheduling schemes according to the demand vector and the idle matrix is ​​as follows:

[0186]

[0187] Among them, s k2 The scheduling plan P k The sub-scheduling scheme score corresponding to the CPU / memory balance, (r c ,r m ) is the demand vector of the task to be scheduled for CPU and memory, (f ck ,f mk ) is the idle matrix of CPU and memory on node k.

[0188] In this embodiment, the problem of load imbalance in the scheduling process can be solved by calculating the sub-scheduling scheme scores corresponding to the CPU / memory balance of each scheduling scheme.

[0189] In detail, the calculating, based on the scheduling task volume of the scheduling platform, a sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the communication indicator includes:

[0190] Obtaining other scheduling tasks in each node in the scheduling platform that have a communication dependency relationship with the task to be scheduled, and obtaining multiple subtask quantities;

[0191] The plurality of subtask amounts are counted to obtain a total task amount; and a sub-scheduling scheme score corresponding to the communication indicator of each scheduling scheme in the plurality of scheduling schemes is calculated according to the total task amount and the plurality of subtask amounts.

[0192] Specifically, the specific method of calculating the sub-scheduling scheme score of each scheduling scheme corresponding to the communication indicator in the multiple scheduling schemes according to the total task amount and the multiple sub-task amounts is as follows:

[0193]

[0194] Among them, s k3 is the scheduling plan P k The sub-scheduling scheme score corresponding to the communication indicator, n is the total number of tasks, n k is the number of subtasks located on node k.

[0195] In this embodiment, by calculating the sub-scheduling scheme scores of the communication indicators corresponding to each scheduling scheme, scheduling failures caused by network communication delays can be reduced, the success rate and reliability of scheduling can be improved, and the efficiency of scheduling can be improved.

[0196] The solution determination module 106 is configured to determine the scheduling solution with the highest solution score as the solution to be scheduled.

[0197] In the embodiment of the present invention, the scheme scores of the various scheduling schemes are compared to determine the scheduling scheme corresponding to the highest scheme score as the scheme to be scheduled.

[0198] The scheduling module 107 is configured to schedule the tasks to be scheduled according to the scheduling plan.

[0199] After determining the to-be-scheduled scheme, the embodiment of the present invention schedules the to-be-scheduled tasks according to the to-be-scheduled scheme.

[0200] When performing task scheduling, resources within the computer are allocated to the tasks to be scheduled according to the scheduling plan so that the tasks to be scheduled can be executed and completed, thereby ensuring the ultimate implementation of the deep learning algorithm on the computer.

[0201] When scheduling tasks, the embodiments of the present invention calculate the solution scores of different scheduling solutions based on multiple scheduling indicators, and then determine the undetermined angle solution. This avoids problems such as the inability to execute a scheduled task or the large resource consumption of a scheduled task affecting the execution of other scheduled tasks that occur when scheduling based on a single scheduling indicator. This improves the reliability and effectiveness of scheduling, thereby increasing scheduling efficiency. Therefore, the task scheduling device proposed by the present invention can achieve the purpose of efficient scheduling.

[0202] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing the task scheduling method of the present invention.

[0203] The electronic device 1 may include a processor 10 , a memory 11 , and a bus, and may further include a computer program stored in the memory 11 and executable on the processor 10 , such as a task scheduler 12 .

[0204] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Furthermore, the memory 11 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the task scheduler 12, etc., but can also be used to temporarily store data that has been output or is to be output.

[0205] In some embodiments, the processor 10 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and circuits. It executes or executes programs or modules stored in the memory 11 (such as executing a task scheduler) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0206] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory 11 and at least one processor 10, etc.

[0207] Figure 3 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 3The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0208] For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for powering the various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device 1 may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0209] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0210] Optionally, the electronic device 1 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0211] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0212] The task scheduling program 12 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve the following:

[0213] Get the tasks to be scheduled on the scheduling platform;

[0214] generating a pairwise comparison matrix of the tasks to be scheduled according to a plurality of scheduling indicators, wherein the scheduling indicators include indicators related to system resources of the scheduling platform and / or indicators related to the tasks to be scheduled;

[0215] Calculating the eigenvalues ​​and eigenvectors of the pairwise comparison matrix;

[0216] Determining weights of the plurality of scheduling indicators according to the eigenvalues ​​and eigenvectors;

[0217] Obtaining multiple scheduling schemes for the task to be scheduled, and calculating scheme scores of the multiple scheduling schemes according to weights of the multiple scheduling indicators;

[0218] Determine the scheduling plan with the highest plan score as the plan to be scheduled;

[0219] The tasks to be scheduled are scheduled according to the scheduling plan.

[0220] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0221] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0222] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0223] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0224] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0225] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a table in a claim should not be construed as limiting the claim.

[0226] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Second-order terms are used to indicate names and do not imply any particular order.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A task scheduling method, characterized in that: The method comprises: Get the tasks to be scheduled on the scheduling platform; Obtaining a GPU concentration, a CPU / memory balance, and a communication index related to the scheduling platform as scheduling indicators, and generating a pairwise comparison matrix for the tasks to be scheduled based on the scheduling indicators, including: determining priorities of multiple scheduling indicators based on attribute information of the tasks to be scheduled, initializing values ​​of designated elements in the pairwise comparison matrix based on the priorities and a table of correspondence between priorities and values, and determining values ​​of other elements in the pairwise comparison matrix other than the designated elements based on a consistency condition; Calculating the eigenvalues ​​and eigenvectors of the pairwise comparison matrix; Determining the weight of each scheduling indicator according to the eigenvalue and the eigenvector; Obtaining multiple scheduling schemes for the tasks to be scheduled; Calculate the sub-scheduling scheme scores corresponding to the GPU concentration and the CPU / memory balance of each scheduling scheme based on the remaining number of GPUs, the remaining number of CPUs, and the amount of memory associated with the scheduling platform; Calculating a sub-scheduling scheme score of each scheduling scheme corresponding to the communication indicator according to the scheduling task volume related to the scheduling platform, including: obtaining other scheduling tasks that have a communication dependency relationship with the task to be scheduled in each node in the scheduling platform to obtain multiple sub-task volumes, counting the multiple sub-task volumes to obtain a total task volume, and calculating a sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the communication indicator according to the total task volume and the multiple sub-task volumes; Performing weight calculation on the calculated scores of the multiple sub-scheduling schemes of each scheduling scheme according to the weights of the respective scheduling indicators to obtain scheme scores of the multiple scheduling schemes; Determine the scheduling plan with the highest plan score as the plan to be scheduled; The tasks to be scheduled are scheduled according to the scheduling plan.

2. The task scheduling method according to claim 1, wherein: Calculating the sub-scheduling scheme scores of each scheduling scheme corresponding to the GPU concentration and the CPU / memory balance according to the remaining number of GPUs, the remaining number of CPUs, and the amount of memory associated with the scheduling platform includes: Calculating a sub-scheduling scheme score corresponding to the GPU concentration for each scheduling scheme in the multiple scheduling schemes according to the remaining number of GPUs in the scheduling platform; A sub-scheduling scheme score corresponding to the CPU / memory balance degree of each scheduling scheme in the multiple scheduling schemes is calculated according to the remaining number of CPUs and memory amounts of the scheduling platform.

3. The task scheduling method according to claim 2, wherein: Calculating the sub-scheduling scheme score of each scheduling scheme in the multiple scheduling schemes corresponding to the GPU concentration according to the remaining number of GPUs in the scheduling platform includes: Determine the number of GPUs required for the task to be scheduled; Obtain a node set in the scheduling platform whose number of remaining GPUs is greater than or equal to the required number of GPUs; Determine a remaining minimum value and a remaining maximum value of the number of remaining GPUs in the node set; A sub-scheduling scheme score corresponding to the GPU concentration of each scheduling scheme in the multiple scheduling schemes is calculated according to the remaining minimum value and the remaining maximum value.

4. The task scheduling method according to claim 2, wherein: The calculating, based on the remaining number of CPUs and memory of the scheduling platform, a sub-scheduling scheme score of each scheduling scheme corresponding to the CPU / memory balance degree in the multiple scheduling schemes includes: Obtain the remaining number of CPUs and memory of each node in the scheduling platform, and generate an idle matrix based on the remaining number of CPUs and memory; Determining a CPU and memory requirement vector of the task to be scheduled based on the attribute information of the task to be scheduled; A sub-scheduling scheme score corresponding to the CPU / memory balance degree of each scheduling scheme in the multiple scheduling schemes is calculated according to the demand vector and the idle matrix.

5. A task scheduling device, characterized in that: The device comprises: The task acquisition module is used to obtain the tasks to be scheduled on the scheduling platform; a matrix generation module, configured to obtain GPU concentration, CPU / memory balance, and communication indicators related to the scheduling platform as scheduling indicators, and generate a pairwise comparison matrix for the tasks to be scheduled based on the scheduling indicators, including: determining priorities of multiple scheduling indicators based on attribute information of the tasks to be scheduled, initializing values ​​of designated elements in the pairwise comparison matrix based on the priorities and a table of correspondence between priorities and values, and determining values ​​of other elements in the pairwise comparison matrix other than the designated elements based on a consistency condition; A feature calculation module, used to calculate the eigenvalues ​​and eigenvectors of the pairwise comparison matrix; A weight determination module, configured to determine the weight of each scheduling indicator based on the eigenvalue and eigenvector; A score calculation module, used to obtain multiple scheduling schemes for the task to be scheduled; Calculate the sub-scheduling scheme scores of each scheduling scheme corresponding to the GPU concentration and the CPU / memory balance according to the remaining number of GPUs, the remaining number of CPUs, and the amount of memory related to the scheduling platform; calculate the sub-scheduling scheme scores of each scheduling scheme corresponding to the communication indicator according to the scheduling task volume related to the scheduling platform, including: obtaining other scheduling tasks that have a communication dependency relationship with the task to be scheduled in each node in the scheduling platform to obtain multiple sub-task volumes, counting the multiple sub-task volumes to obtain a total task volume, and calculating the sub-scheduling scheme scores of each scheduling scheme in the multiple scheduling schemes corresponding to the communication indicator according to the total task volume and the multiple sub-task volumes; The score calculation module is further configured to perform weight calculation on the calculated scores of the multiple sub-scheduling schemes of each scheduling scheme according to the weights of the respective scheduling indicators to obtain scheme scores of the multiple scheduling schemes; A scheme determination module is used to determine the scheduling scheme with the highest scheme score as the scheme to be scheduled; The scheduling module is used to schedule the tasks to be scheduled according to the scheduling plan.

6. An electronic device, characterized in that: The electronic device comprises: a memory storing at least one computer program instruction; and A processor is configured to execute computer program instructions stored in the memory to perform the task scheduling method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the task scheduling method according to any one of claims 1 to 4 is implemented.

Images