A task scheduling method, apparatus, medium, and equipment

Abstract

This specification discloses a task scheduling method, apparatus, medium, and device. The method involves: acquiring task description information of a computational task; inputting the task description information into a pre-trained first prediction model to obtain a first execution time for the computational task output by the first prediction model; determining a resource scheduling strategy for the computational task based on the task description information and the first execution time; executing the computational task based on the resource scheduling strategy and the task description information; determining resource utilization information during the execution of the computational task and determining a second execution time for the computational task based on the resource utilization information; adjusting the resource scheduling strategy for the computational task based on the second execution time, and continuing to execute the computational task according to the adjusted resource scheduling strategy. By dynamically adjusting the resource scheduling strategy for the computational task, the utilization efficiency of computing resources and the execution efficiency of the computational task are improved.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of computer technology, and particularly relates to a task scheduling method and device, medium and equipment. BACKGROUND

[0002] At present, with the rapid development of technologies such as big data, cloud computing and artificial intelligence, a large-scale intelligent computing cluster (hereinafter referred to as a cluster) composed of thousands of central processing units (CPUs), graphic processing units (GPUs) and other special hardware accelerators and other computing resources can provide powerful computing power for data centers or cloud computing platforms and the like to process various computing tasks.

[0003] However, with the expansion of the size of the cluster and the diversification of the types of computing tasks, how to effectively manage and schedule the vast computing resources in the cluster to ensure that the computing tasks can be completed within a reasonable time while maximizing the utilization of computing resources has become a challenge.

[0004] The existing methods of managing computing resources and scheduling computing tasks in a cluster cannot accurately predict the execution time of a computing task. If the predicted execution time is too long, it may lead to the allocation of too many computing resources, resulting in resource waste and subsequent computing task execution delays. If the predicted execution time is too short, it may lead to computing task execution failure or increased competition for computing resources.

[0005] Therefore, the present specification provides a task scheduling method, device, medium and equipment. SUMMARY

[0006] The present specification provides a task scheduling method, device, medium and equipment to partially solve the above problems existing in the prior art.

[0007] The present specification adopts the following technical solutions:

[0008] The present specification provides a task scheduling method, comprising:

[0009] obtaining task description information of a computing task;

[0010] inputting the task description information into a pre-trained first prediction model to obtain a first execution time of the computing task output by the first prediction model;

[0011] determining a resource scheduling strategy of the computing task according to the task description information and the first execution time;

[0012] executing the computing task according to the resource scheduling strategy and the task description information;

[0013] determining resource utilization information in the execution of the computing task, and determining a second execution time of the computing task according to the resource utilization information;

[0014] adjusting a resource scheduling strategy of the computing task according to the second execution time, so as to continue executing the computing task according to the adjusted resource scheduling strategy.

[0015] Optionally, the task description information is input into a pre-trained first prediction model to obtain a first execution time of the computing task output by the first prediction model, and specifically includes:

[0016] performing feature extraction on the task description information to determine features of the task description information;

[0017] determining a pre-trained first prediction model matched with the computing task according to the features of the task description information;

[0018] inputting the features of the task description information into the first prediction model to obtain the first execution time of the computing task output by the first prediction model.

[0019] Optionally, the first prediction model is trained, and specifically includes:

[0020] obtaining task description information of each historical computing task and actual execution time of each historical computing task;

[0021] performing feature extraction on the task description information of each historical computing task to determine features of each historical computing task;

[0022] clustering the features of each historical computing task to determine each clustering result;

[0023] for each clustering result, selecting target features from the clustering result, and inputting the target features into a first prediction model to be trained to obtain a prediction result output by the first prediction model to be trained;

[0024] determining a loss value according to a difference between the prediction result and actual execution time of a historical computing task corresponding to the target features, and training the first prediction model to be trained according to the loss value, wherein the difference is positively correlated with the loss value.

[0025] Optionally, the second execution time of the computing task is determined according to the resource utilization information, and specifically includes:

[0026] performing feature extraction on the resource utilization information and the task description information to obtain features of the computing task;

[0027] inputting the features of the computing task into a pre-trained second prediction model to obtain a second execution time output by the second prediction model.

[0028] Optionally, the second prediction model is trained, and specifically includes:

[0029] obtaining description information of a historical computing task and resource utilization information of the historical computing task;

[0030] extracting features of the historical computing task from the description information and the resource utilization information of the historical computing task;

[0031] inputting the features of the historical computing task into a second prediction model to be trained to obtain a prediction result output by the second prediction model to be trained;

[0032] determining a loss value according to a difference between the prediction result and an actual execution time of the historical computing task, and training the second prediction model to be trained according to the loss value, wherein the difference is positively correlated with the loss value.

[0033] Optionally, according to the second execution time, a resource scheduling strategy of the computing task is adjusted, so that the computing task is continued to be executed according to the adjusted resource scheduling strategy, and specifically includes:

[0034] in a case where the second execution time is greater than a first execution time, determining computing resources that need to be continuously allocated to the computing task and a use time of the computing resources as increase resource information of the computing task;

[0035] if it is determined that there is no idle computing resource that meets the increase resource information, determining a second execution time of other computing tasks according to resource utilization information collected in execution processes of the other computing tasks, and determining a computing task that needs to reduce computing resources as a resource abundant task according to the second execution time of the other computing tasks;

[0036] from the resource abundant tasks, determining a resource abundant task whose reduced computing resources and use time of the computing resources meet the increase resource information of the computing task as a target task;

[0037] taking description information of a part of the computing task that needs to be executed according to the increase resource information as first description information, and taking description information of a remaining part of the computing task other than the part as second description information;

[0038] From the computing resources required to perform the target task, determine the computing resources matching the computing resources in the increased resource information and perform the partial task corresponding to the first description information, and continue to perform the remaining partial task according to the resource scheduling strategy and the second description information.

[0039] Optionally, according to the task description information and the first execution time, determine the resource scheduling strategy of the computing task, specifically including:

[0040] According to the task description information and the first execution time, determine the initial resource scheduling strategy of the computing task;

[0041] According to the computing resources contained in the initial resource scheduling strategy and the use time of the computing resources, determine the execution efficiency value of executing the computing task according to the initial resource scheduling strategy;

[0042] Maximizing the execution efficiency value as the optimization goal, adjust the computing resources contained in the initial resource scheduling strategy and the use time of the computing resources, to determine the resource scheduling strategy of the computing task during execution.

[0043] The specification provides a task scheduling device, comprising:

[0044] An acquisition module is configured to acquire task description information of a computing task;

[0045] An input module is configured to input the task description information into a first prediction model trained in advance, to obtain a first execution time of the computing task output by the first prediction model;

[0046] A first determination module is configured to determine a resource scheduling strategy of the computing task according to the task description information and the first execution time;

[0047] An execution module is configured to execute the computing task according to the resource scheduling strategy and the task description information;

[0048] A second determination module is configured to determine resource utilization information in the execution process of the computing task, and determine a second execution time of the computing task according to the resource utilization information;

[0049] An adjustment module is configured to adjust the resource scheduling strategy of the computing task according to the second execution time, to continue to execute the computing task according to the adjusted resource scheduling strategy.

[0050] The specification provides a computer readable storage medium, the storage medium stores a computer program, the computer program is executed by a processor to realize the above task scheduling method.

[0051] The specification provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements a task scheduling method when executing the program.

[0052] The above at least one technical solution adopted by the specification can achieve the following beneficial effects:

[0053] The task scheduling method provided by the specification first acquires task description information of a computing task. The task description information is input into a first prediction model trained in advance to obtain a first execution time of the computing task output by the first prediction model. According to the task description information and the first execution time, a resource scheduling strategy of the computing task is determined. According to the resource scheduling strategy and the task description information, the computing task is executed. Resource utilization information in the execution process of the computing task is determined, and according to the resource utilization information, a second execution time of the computing task is determined. According to the second execution time, the resource scheduling strategy of the computing task is adjusted, so as to continue to execute the computing task according to the adjusted resource scheduling strategy.

[0054] Through the first prediction model, the execution time of the computing task is predicted when the computing task has not been executed, so as to allocate suitable computing resources for the computing task and perform task scheduling. During the execution process of the computing task, the execution time of the computing task is determined again according to the resource utilization information in the execution process of the computing task, and the resource scheduling strategy of the computing task is dynamically adjusted, so as to improve the utilization efficiency of the computing resources and the execution efficiency of the computing task. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are included to provide a further understanding of the specification, constitute a part of the specification, and the illustrative embodiments of the specification and their description serve to explain the specification, and do not constitute an improper limitation on the specification. In the drawings:

[0056] Figure 1 A flowchart of a task scheduling method provided by an embodiment of the specification is shown in the figure;

[0057] Figure 2 A large-scale intelligent computing cluster resource scheduling diagram provided by the specification is shown in the figure;

[0058] Figure 3 A flowchart of a large-scale intelligent computing cluster resource scheduling provided by the specification is shown in the figure;

[0059] Figure 4 A schematic diagram of a task scheduling device provided by the specification is shown in the figure;

[0060] Figure 5 A schematic diagram of a task scheduling device provided by the specification is shown in the figure; Figure 1A structural schematic diagram of an electronic device. DETAILED DESCRIPTION

[0061] For the purposes of the present description, the technical solutions and advantages, the technical solutions of the present description will be described in detail below with reference to the specific embodiments of the present description and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present description, not all embodiments. Based on the embodiments in the present description, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0062] The technical solutions provided by the embodiments of the present description will be described in detail below with reference to the drawings.

[0063] Figure 1 A flowchart of a task scheduling method provided by an embodiment of the present description includes the following steps:

[0064] S100: Obtain task description information of a computing task.

[0065] In the present description, the process of performing execution time prediction and resource scheduling is performed by the server in the embodiments of the present description. Of course, the present description does not limit the device that performs the process of performing execution time prediction and resource scheduling, and devices such as personal computers and mobile terminals can also be used to perform execution time prediction and resource scheduling. In order to facilitate description, the following describes the server (hereinafter referred to as the server) as the execution subject for controlling a large-scale intelligent computing cluster.

[0066] In one or more embodiments of the present description, the server obtains the task description information of the computing task to be executed. The task description information includes the computing task ID, the number of GPUs required by the computing task, the number of CPUs required by the computing task, the task creation time, the task start command, the task parallel computing mode, and the like.

[0067] S102: Input the task description information into a pre-trained first prediction model to obtain a first execution time of the computing task output by the first prediction model.

[0068] In one or more embodiments of the present description, the server inputs the obtained description information of the computing task into a pre-trained first prediction model to obtain a first execution time of the computing task output by the first prediction model.

[0069] Specifically, the server extracts features of the task description information, determines the features of the task description information, and determines a pre-trained first prediction model matched with the computing task according to the features of the task description information. Then, the server can input the features of the task description information into the first prediction model to obtain a first execution time of the computing task output by the first prediction model.

[0070] The method for training the first prediction model comprises: the server obtaining task description information of each historical computing task and an actual execution time of each historical computing task. Then, the server extracts features of the task description information of each historical computing task. The server can cluster the features of each historical computing task to determine each clustering result. Then, for each clustering result, the server selects a target feature from the clustering result and inputs the target feature into the first prediction model to be trained to obtain a prediction result output by the first prediction model to be trained. Then, the server can determine a loss value according to a difference between the prediction result and the actual execution time of the historical computing task corresponding to the target feature, and train the first prediction model to be trained according to the loss value, wherein the difference is positively correlated with the determined loss value.

[0071] It is worth noting that in the present specification, the form of the task description information obtained by the server is not limited, and the task description information can be in various forms. Therefore, when extracting features of the task description information, if the task description information is an audio information, the audio information can be converted into a text form of the task description information through voice recognition technology. If the task description information is image data, the task description information in the image data can be extracted through image recognition technology to convert the image data into a text form of the task description information. Of course, the task description information obtained by the server can also be a text form of the task description information, and no additional text conversion step is required. Then, the server extracts vectors from the text in the task description information, such as task ID, GPU quantity and CPU quantity, through a word vector model (such as Word2Vec) to obtain a feature vector matrix of the task description information. The server then clusters the feature vector matrix through a clustering algorithm, which is not limited in the present specification, such as a K-means (KMEANS) algorithm or a density-based spatial clustering (Density-Based Spatial Clustering of Applications with Noise, DBSCAN) algorithm.

[0072] S104: determining a resource scheduling strategy of the computing task according to the task description information and the first execution time.

[0073] S106: Perform the computing task according to the resource scheduling strategy and the task description information.

[0074] In one or more embodiments of the present specification, the server determines the resource scheduling strategy of the computing task according to the determined first execution time and the task description information of the computing task. For example, the first execution time of the computing task is long, and according to the information such as the number of GPUs or the number of CPUs required by the computing task described in the task description information, in order to ensure the efficient execution of the computing task and maximize the utilization of computing resources, the server determines the resource scheduling strategy to allocate more computing resources and use time of the computing resources to the computing task to meet the long first execution time and ensure the smooth execution of the computing task. Or, the server determines that the first execution time of the computing task is 6 hours at dawn. Since the network bandwidth and other computing resources are artificially occupied less during the night, more computing resources can be allocated to the computing task to speed up the completion of the computing task, and there is no need to delay the completion of the computing task to the daytime, avoiding the competition for computing resources of the large-scale intelligent computing cluster in the daytime.

[0075] Then, the computing resources can be allocated to the computing task according to the resource scheduling strategy, and the computing task can be executed according to the task description information. It is worth noting that the computing resources are computing devices including GPUs, CPUs, and other special hardware accelerators in the large-scale intelligent computing cluster.

[0076] S108: Determine the resource utilization information in the execution process of the computing task, and determine the second execution time of the computing task according to the resource utilization information.

[0077] S110: Adjust the resource scheduling strategy of the computing task according to the second execution time, and continue to execute the computing task according to the adjusted resource scheduling strategy.

[0078] In one or more embodiments of the present specification, in order to maximize the utilization of computing resources in the large-scale intelligent computing cluster and dynamically adjust the use of computing resources of the computing task, the computing task is better executed. In the process of executing the computing task, the server collects the resource utilization information when the computing task is executed, and determines the second execution time of the computing task according to the resource utilization information.

[0079] Specifically, the server extracts features from the resource utilization information and the task description information to obtain the features of the computing task. Then, the features of the computing task are input into the second prediction model pre-trained to obtain the second execution time output by the second prediction model.

[0080] The resource utilization information can be GPU utilization, CPU utilization, memory utilization, and the like, and is mainly used to indicate the utilization of computing resources. The second execution time is the execution time of the computing task evaluated according to the utilization of computing resources of the computing task, that is, the server actually executes the computing task for a period of time according to the resource scheduling strategy, and then according to the utilization of computing resources in the execution process, the execution time of the computing task is predicted again to obtain the time length. It should be noted that the resource utilization information determined by the server can also be in various forms. When the resource utilization information is extracted, if the resource utilization information is an audio information, the audio information can be converted into text form resource utilization information through voice recognition technology. If the resource utilization information is image data, the resource utilization information in the image data can be extracted through image recognition technology to realize the conversion of the image data into text form resource utilization information. Of course, the resource utilization information determined by the server can also be in text form and does not need to be converted into text form. Therefore, the feature extraction of the resource utilization information can refer to the above-mentioned feature extraction method of the task description information.

[0081] Then, the server can adjust the resource scheduling strategy of the computing task according to the second execution time of the computing task. When the second execution time is the time required for the overall completion of the computing task, if the second execution time is greater than the first execution time, it indicates that the computing resources allocated to the computing task according to the resource scheduling strategy determined by the first execution time are not enough, and therefore more computing resources need to be allocated to the computing task to meet its needs. If the second execution time is less than or equal to the first execution time, it indicates that the computing resources allocated to the computing task according to the resource scheduling strategy determined by the first execution time are sufficient, and therefore the resource scheduling strategy is not adjusted. Therefore, in this specification, not adjusting the resource scheduling strategy can also be regarded as a way of adjusting the resource scheduling strategy, that is, zero adjustment. Of course, the second execution time can also be the remaining execution time of the computing task, which is not limited in this specification.

[0082] In this specification, the method of training the second prediction model can be that the server obtains the description information of the historical computing task and the resource utilization information of the historical computing task. Then, the server can extract the features of the historical computing task from the description information and the resource utilization information, and input the features of the historical computing task into the second prediction model to be trained to obtain the prediction result output by the second prediction model to be trained. In the subsequent process, the server can determine the loss value according to the difference between the prediction result and the actual execution time of the historical computing task, and train the second prediction model to be trained according to the loss value, wherein the difference is positively correlated with the loss value.

[0083] Furthermore, in one or more embodiments of this specification, if the second execution time is longer than the first execution time, it means that the computing resources allocated to the computing task in the resource scheduling strategy determined based on the first execution time are insufficient. Therefore, in order to ensure that the computing task can be executed smoothly and will not stop due to a lack of computing resources, thereby improving the execution efficiency of the computing task, the server needs to further determine the computing resources that the computing task needs to be allocated and the usage time of the computing resources that need to be allocated, as additional resource information for the computing task.

[0084] If the server determines that a large-scale intelligent computing cluster cannot continue to allocate computing resources to computing tasks due to the lack of idle computing resources that meet the requirements for increasing resource information, it needs to determine the second execution time of other computing tasks based on the resource utilization information collected during the execution of other computing tasks. Based on the second execution time of other computing tasks, it then identifies computing tasks that require reduced computing resources and designates them as resource-sufficient tasks. That is, other computing tasks whose second execution time is shorter than their first execution time are considered resource-sufficient tasks. Therefore, the resource-sufficient task mentioned here refers to a task for which the server's resource scheduling strategy allocates computing resources within the first execution time that are not only sufficient for the task to complete on time within the first execution time, but also have computing resources available to execute other computing tasks without affecting the timely completion of the task itself.

[0085] The server can then identify the resource-sufficient tasks from among the resource-sufficient tasks that require reduced computing resources and sufficient usage time to meet the increased resource information requirements, and designate these as target tasks. The server will then use the description information of the portion of the computing task that requires the increased resource information as the first description information, and the description information of the remaining tasks excluding those requiring increased resource information as the second description information.

[0086] Finally, the server can determine the computing resources that match the computing resources in the increased resource information from the computing resources required to execute the target task, execute the part of the task corresponding to the first description information, and continue to execute the remaining part of the task according to the resource scheduling policy and the second description information.

[0087] Even when there are no idle computing resources required for computing tasks in a large-scale intelligent computing cluster, it can still make use of the allocated computing resources, thereby maximizing the utilization of computing resources in the large-scale intelligent computing cluster and ensuring that computing tasks are completed within a reasonable execution time, thus improving the execution efficiency of computing tasks.

[0088] In one or more embodiments of the present specification, in order to further ensure that the determined resource scheduling strategy can ensure efficient execution of the computing task under the condition of maximizing the utilization of computing resources, the server can also determine the initial resource scheduling strategy of the computing task according to the task description information and the first execution time. Then, the server determines the execution efficiency value of executing the computing task according to the computing resources contained in the initial resource scheduling strategy and the use time of the computing resources. The execution efficiency value can refer to the resource utilization information of the historical computing task that has the same computing resources as the computing task, so as to judge the execution efficiency value of the computing task.

[0089] The execution efficiency value is maximized as the optimization target, and the computing resources contained in the initial resource scheduling strategy and the use time of the computing resources are adjusted to determine the resource scheduling strategy of the computing task during execution. This process can be understood as determining the maximum execution efficiency value according to the resource utilization information of the historical computing task that has the same computing resources as the computing task, and determining the resource scheduling strategy of the computing task according to the adjusted resource scheduling strategy of the historical computing task as a reference.

[0090] Figure 2 A large-scale intelligent computing cluster resource scheduling schematic diagram is provided for the present specification. Figure 2 The large-scale intelligent computing cluster includes a front-end task receiving and prediction module, a database and task monitoring module, and a back-end task starting and scheduling module. Among them, the server receives the task description file of the computing task submitted by the user through the front-end task receiving and static prediction module, and parses it into task description information, and then inputs the first prediction model to obtain the predicted execution time. The server submits the predicted execution time and the task description file to the database and task monitoring module to form the task complete information, stores it in the database, and scans the database by the process monitoring to detect the computing task submitted to the database. Then, the server detects the occurrence of the computing task through the database and task monitoring module, and sends the task complete information to the back-end task starting and scheduling module. Then, the server starts the computing task through the back-end task starting and scheduling module, sends the task complete information to the scheduling and optimizer, determines the resource scheduling strategy, and executes the computing task according to the resource scheduling strategy. During the execution of the computing task, the server detects the resource utilization information through the back-end task starting and scheduling module and stores it in the database, and predicts the new execution time through the second prediction model according to the resource utilization information and the task complete information. Finally, the server adjusts the resource scheduling strategy according to the new execution time and continues to execute the computing task.

[0091] Figure 3 A large-scale intelligent computing cluster resource scheduling process schematic diagram is provided for the present specification. As Figure 3As shown, the server receives the task description information through the front-end task receiving and prediction module, determines the task description information, and predicts the execution time of the computing task through the first prediction model to determine the predicted value. Then, the server sends the task description information and the predicted value to the database and the task monitoring module. When the database and the task monitoring module detect a new task, the server sends the task description information and the predicted value to the back-end task starting and scheduling module to start the computing task. Through the back-end task starting and scheduling module, the server sends the task description information and the predicted value, i.e., the initial static information of the task, to the scheduling and optimizer to determine the initial resource scheduling strategy, and executes the computing task according to the initial resource scheduling strategy. During the execution of the computing task, the server detects the resource utilization information through the back-end task starting and scheduling module, and predicts a new execution time (task execution time adjustment value) through the second prediction model according to the resource utilization information. Finally, the server determines whether to adjust the resource scheduling strategy according to the new execution time. If not, the computing task is continued to be executed. If yes, the secondary scheduling of the computing resource is performed to determine the resource scheduling strategy.

[0092] In summary, the task scheduling method provided by the present specification predicts the execution time of the computing task through the first prediction model before the computing task is executed, so as to allocate appropriate computing resources for the computing task and perform task scheduling. During the execution of the computing task, the execution time of the computing task is determined again according to the resource utilization information of the computing task, the resource scheduling strategy of the computing task is dynamically adjusted, the utilization efficiency of the computing resource is improved, and the execution efficiency of the computing task is improved. Moreover, when there is no idle computing resource required by the computing task in the large-scale intelligent computing cluster, the allocated computing resource can still be transferred, so that the utilization rate of the computing resource in the large-scale intelligent computing cluster is maximized, and the execution efficiency of the computing task is improved. Meanwhile, the resource scheduling strategy of the computing task is determined by referring to the adjusted resource scheduling strategy of the historical computing task, so as to maximize the execution efficiency of the computing task.

[0093] Based on the same idea, the present specification also provides a corresponding task scheduling device, as shown in Figure 4 .

[0094] Figure 4 The task scheduling device provided by the present specification includes:

[0095] The obtaining module 400 is configured to obtain task description information of a computing task.

[0096] The input module 402 is configured to input the task description information into a pre-trained first prediction model to obtain a first execution time of the computing task output by the first prediction model.

[0097] The first determination module 404 is configured to determine a resource scheduling strategy of the computing task according to the task description information and the first execution time.

[0098] The execution module 406 is configured to execute the computing task according to the resource scheduling strategy and the task description information.

[0099] The second determination module 408 is configured to determine resource utilization information in the execution process of the computing task, and determine a second execution time of the computing task according to the resource utilization information.

[0100] The adjustment module 410 is configured to adjust the resource scheduling strategy of the computing task according to the second execution time, and continue to execute the computing task according to the adjusted resource scheduling strategy.

[0101] Optionally, the input module 402 is configured to perform feature extraction on the task description information to determine a feature of the task description information, determine a pre-trained first prediction model matched with the computing task according to the feature of the task description information, input the feature of the task description information into the first prediction model to obtain the first execution time of the computing task output by the first prediction model.

[0102] Optionally, the input module 402 is configured to obtain task description information of each historical computing task and an actual execution time of each historical computing task, perform feature extraction on the task description information of each historical computing task to determine a feature of each historical computing task, cluster the features of the historical computing tasks to determine each clustering result, select a target feature from each clustering result, input the target feature into a first prediction model to be trained to obtain a prediction result output by the first prediction model to be trained, determine a loss value according to a difference between the prediction result and the actual execution time of the historical computing task corresponding to the target feature, and train the first prediction model to be trained according to the loss value, where the difference is positively correlated with the loss value.

[0103] Optionally, the second determination module 408 is configured to perform feature extraction on the resource utilization information and the task description information to obtain a feature of the computing task, input the feature of the computing task into a pre-trained second prediction model to obtain a second execution time output by the second prediction model.

[0104] Optionally, the second determining module 408 is configured to acquire description information of a historical computing task and resource utilization information of the historical computing task, perform feature extraction on the description information and the resource utilization information of the historical computing task, determine a feature of the historical computing task, input the feature of the historical computing task into a second prediction model to be trained, obtain a prediction result output by the second prediction model to be trained, determine a loss value according to a difference between the prediction result and an actual execution time of the historical computing task, and train the second prediction model to be trained according to the loss value, where the difference is positively correlated with the loss value.

[0105] Optionally, the adjusting module 410 is configured to, in a case where the second execution time is greater than the first execution time, determine, as increase resource information of the computing task, computing resources that need to be continuously allocated and a use time of the computing resources that need to be continuously allocated, determine, according to resource utilization information collected in an execution process of another computing task, a second execution time of the another computing task, and determine, according to the second execution time of the another computing task, a computing task that needs to reduce computing resources as a resource abundant task, determine, from the resource abundant tasks, a resource abundant task that has reduced computing resources and a use time of the computing resources satisfying the increase resource information of the computing task as a target task, determine, as first description information, description information of a partial task of the computing task that needs to be executed according to the increase resource information, and determine, as second description information, description information of a remaining partial task of the computing task excluding the partial task, determine, from computing resources required for executing the target task, computing resources matching the increase resource information in the computing resources and execute the partial task corresponding to the first description information, and continue to execute the remaining partial task according to the resource scheduling strategy and the second description information.

[0106] Optionally, the first determining module 404 is configured to determine, according to the task description information and the first execution time, an initial resource scheduling strategy of the computing task, determine an execution efficiency value of executing the computing task according to computing resources included in the initial resource scheduling strategy and a use time of the computing resources, and maximize the execution efficiency value as an optimization target to adjust the computing resources included in the initial resource scheduling strategy and the use time of the computing resources, so as to determine a resource scheduling strategy of the computing task in execution.

[0107] The specification also provides a computer readable storage medium storing a computer program, where the computer program can be used to execute the above method. Figure 1A task scheduling method is provided.

[0108] The specification also provides Figure 5 A schematic structural diagram of an electronic device is shown. As Figure 5 As shown at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs to implement the above Figure 1 The task scheduling method.

[0109] Of course, in addition to the software implementation, the specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or a logic device.

[0110] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0111] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0112] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0113] For the sake of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of each unit can be implemented in one or more software and / or hardware in the implementation of the present specification.

[0114] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, the present specification can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0115] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0116] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. ​ The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0118] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0119] The memory can include non-persistent memory and / or persistent memory, such as flash memory, read-only memory (ROM), and / or volatile / non-volatile random access memory (RAM), among others. The memory is an example of computer-readable media.

[0120] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0121] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0122] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0124] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different aspects of the description. For each embodiment, the description focuses on the differences from the other embodiments. Each embodiment is to be read in isolation, with the understanding that the same or similar features from other embodiments can be combined with the features of the respective embodiment. In particular, the description of the system embodiments is kept relatively short, as the system embodiments are largely analogous to the method embodiments.

[0125] The above description is embodied in the form of only a description of embodiments of the present specification, and is not intended to limit the present specification. Various changes and modifications can be made by those skilled in the art based on the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A task scheduling method, characterized by, The method comprises: obtaining task description information of a computing task; inputting the task description information into a pre-trained first prediction model to obtain a first execution time of the computing task output by the first prediction model; determining a resource scheduling strategy of the computing task according to the task description information and the first execution time; executing the computing task according to the resource scheduling strategy and the task description information; determining resource utilization information in the execution process of the computing task, and determining a second execution time of the computing task according to the resource utilization information; adjusting the resource scheduling strategy of the computing task according to the second execution time, and continuing to execute the computing task according to the adjusted resource scheduling strategy, wherein, in the case that the second execution time is greater than the first execution time, the computing resources that need to be continuously allocated and the use time of the computing resources that need to be continuously allocated of the computing task are determined as increase resource information of the computing task; if it is determined that there is no idle computing resource that meets the increase resource information, the second execution time of other computing tasks is determined according to the collected resource utilization information in the execution process of the other computing tasks, and a computing task that needs to reduce computing resources is determined as a resource abundant task according to the second execution time of the other computing tasks; from each resource abundant task, a resource abundant task whose reduced computing resources and use time of the computing resources meet the increase resource information of the computing task is determined as a target task; the description information of a part of the computing task that needs to be executed according to the increase resource information is determined as first description information, and the description information of the remaining part of the computing task other than the part of the computing task is determined as second description information; from the computing resources required to execute the target task, the computing resources that match the computing resources in the increase resource information are determined and the part of the computing task corresponding to the first description information is executed, and the remaining part of the computing task is continued to be executed according to the resource scheduling strategy and the second description information.

2. The method of claim 1, wherein, The method comprises: extracting features of the task description information to determine the features of the task description information; determining a pre-trained first prediction model matched with the computing task according to the features of the task description information; inputting the features of the task description information into the first prediction model to obtain the first execution time of the computing task output by the first prediction model.

3. The method of claim 2, wherein, The method comprises: obtaining task description information of each historical computing task and actual execution time of the historical computing task; extracting features of the task description information of each historical computing task to determine the features of each historical computing task; clustering the features of each historical computing task to determine each clustering result; For each clustering result, a target feature is selected from the clustering result, and the target feature is input into a first prediction model to be trained to obtain a prediction result output by the first prediction model to be trained; A loss value is determined according to a difference between the prediction result and an actual execution time of a historical computing task corresponding to the target feature, and the first prediction model to be trained is trained according to the loss value, wherein the difference is positively correlated with the loss value.

4. The method of claim 1, wherein, According to the resource utilization information, a second execution time of the computing task is determined, specifically including: The resource utilization information and the task description information are subjected to feature extraction to obtain a feature of the computing task; The feature of the computing task is input into a second prediction model pre-trained to obtain a second execution time output by the second prediction model.

5. The method of claim 4, wherein, Training the second prediction model specifically includes: Obtaining description information of a historical computing task and resource utilization information of the historical computing task; The description information of the historical computing task and the resource utilization information are subjected to feature extraction to determine a feature of the historical computing task; The feature of the historical computing task is input into a second prediction model to be trained to obtain a prediction result output by the second prediction model to be trained; A loss value is determined according to a difference between the prediction result and an actual execution time of the historical computing task, and the second prediction model to be trained is trained according to the loss value, wherein the difference is positively correlated with the loss value.

6. The method of claim 1, wherein, According to the task description information and the first execution time, a resource scheduling strategy of the computing task is determined, specifically including: According to the task description information and the first execution time, an initial resource scheduling strategy of the computing task is determined; According to computing resources contained in the initial resource scheduling strategy and usage times of the computing resources, an execution efficiency value of executing the computing task according to the initial resource scheduling strategy is determined; With maximizing the execution efficiency value as an optimization objective, the computing resources contained in the initial resource scheduling strategy and the usage times of the computing resources are adjusted to determine a resource scheduling strategy of the computing task when the computing task is executed.

7. A task scheduling apparatus characterized by comprising: Including: An obtaining module is configured to obtain task description information of a computing task; An input module is configured to input the task description information into a first prediction model pre-trained to obtain a first execution time of the computing task output by the first prediction model; A first determining module is configured to determine a resource scheduling strategy of the computing task according to the task description information and the first execution time; An execution module is configured to execute the computing task according to the resource scheduling strategy and the task description information; A second determining module is configured to determine resource utilization information in an execution process of the computing task and determine a second execution time of the computing task according to the resource utilization information; An adjusting module is configured to adjust a resource scheduling strategy of the computing task according to the second execution time, so as to continue to execute the computing task according to the adjusted resource scheduling strategy. In a case where the second execution time is greater than the first execution time, the computing task requiring to continue to allocate computing resources and the usage time of the computing resources requiring to continue to allocate are determined as the increased resource information of the computing task. If it is determined that there is no idle computing resource meeting the increased resource information, the second execution time of other computing tasks in a process of collecting resource utilization information of the other computing tasks is determined, and a computing task requiring to reduce computing resources is determined as a resource abundant task according to the second execution time of the other computing tasks. From each resource abundant task, a resource abundant task whose reduced computing resources and usage time of the computing resources meet the increased resource information of the computing task is determined as a target task. Description information of a partial task of the computing task required to be executed according to the increased resource information is determined as first description information, and description information of a remaining partial task of the computing task except the partial task is determined as second description information. From computing resources required to execute the target task, computing resources matching the increased resource information are determined and the partial task corresponding to the first description information is executed, and the remaining partial task is continued to be executed according to the resource scheduling strategy and the second description information.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1-6.

Images