Task scheduling method and device
By generating the status space of the scheduling platform and evaluating the optimal status value, dynamically adjusting the scheduling strategy, the complexity of task scheduling in the data asset platform is solved, task execution efficiency and system stability are improved, and resource rational utilization and efficient scheduling are achieved.
Patent Information
- Application Number
- CN202410989139.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-22
AI Technical Summary
In the data asset platform, as the amount of data increases, the dependencies between tasks are complex, and existing scheduling methods are difficult to achieve efficient and stable task scheduling, resulting in high computational complexity, low efficiency and error-prone problems.
By obtaining the task scheduling set and available resource sets, the state space of the scheduling platform is generated based on this information, the state optimal value of each scheduling state is evaluated, and the scheduling strategy is dynamically adjusted to adapt to complex and changeable scenarios, so as to achieve load balancing and rational use of resources.
Improve task execution efficiency, system performance and stability, reduce task waiting time and execution time, improve system throughput and response speed, realize reasonable allocation and efficient utilization of resources, and enhance the flexibility and adaptability of the scheduling platform. .
Smart Images

Figure CN119003114A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a task scheduling method and device. Background Art
[0002] In the data asset platform, reasonable scheduling of tasks in the data asset platform according to business needs and resource conditions can enable the data asset platform to operate efficiently and stably. However, with the continuous growth of enterprise data volume, more and more tasks need to be processed, and the dependency relationship between tasks is complex and error-prone. Therefore, in order to ensure the efficient and stable operation of the data resource platform, it is very important to realize task scheduling. Summary of the invention
[0003] The present disclosure provides a task scheduling method and device to solve one of the technical problems in the related art at least to a certain extent. The technical solution of the present disclosure is as follows:
[0004] According to the first aspect of an embodiment of the present disclosure, a task scheduling method is provided, which is applied to a scheduling platform, and includes: obtaining a task scheduling set and an available resource set; wherein the task scheduling set includes at least one task to be scheduled and metadata information of the at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource; obtaining a state space of the scheduling platform; wherein the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate system state information of the scheduling platform under multiple ordered scheduling states; evaluating the state optimal value of each scheduling state in turn according to the system state information of each scheduling state; and scheduling the at least one task to be scheduled based on the state optimal value of each scheduling state.
[0005] According to a second aspect of an embodiment of the present disclosure, a task scheduling device is provided, which is applied to a scheduling platform, including: a first acquisition module, used to acquire a task scheduling set and an available resource set; wherein the task scheduling set includes at least one task to be scheduled and metadata information of the at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource; a second acquisition module, used to acquire a state space of the scheduling platform; wherein the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate system state information of the scheduling platform under multiple ordered scheduling states; an evaluation module, used to evaluate the state optimal value of each scheduling state according to the system state information of each scheduling state in turn; and a scheduling module, used to schedule the at least one task to be scheduled based on the state optimal value of each scheduling state.
[0006] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the task scheduling method as described in the embodiment of the first aspect of the present disclosure.
[0007] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the task scheduling method as described in the embodiment of the first aspect of the present disclosure.
[0008] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including: a computer program, which, when executed by a processor, implements the task scheduling method as described in the embodiment of the first aspect of the present disclosure.
[0009] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:
[0010] In the present technical scheme, scheduling planning is performed based on the task scheduling set and the available resource set to obtain the state space of the scheduling platform, and the state optimal value of each scheduling state is evaluated in turn according to the system state information indicated by the state space of each scheduling state, so as to schedule at least one task to be scheduled based on the state optimal value of each scheduling state. Therefore, the task scheduling method based on the state optimal value realizes load balancing and rational utilization of resources, improves the execution efficiency of tasks, the performance and stability of the system, and in the case of complex and changeable scheduling scenarios, the task scheduling method based on the state optimal value realizes dynamic adjustment of scheduling strategies to adapt to complex and changeable scheduling scenarios, and improves the flexibility and adaptability of the scheduling platform. Among them, when scheduling at least one task to be scheduled based on the state optimal value of each scheduling state, the target subpath of each scheduling state is determined by considering the state optimal value of each scheduling state, so that the task obtains the required resources at the best time, thereby maximizing the system The overall performance of the system is improved, the waiting time and execution time of tasks are reduced, the system throughput and response speed are improved, and the target scheduling scheme is generated based on the target sub-path of each scheduling state, which comprehensively considers the system resources and task requirements, realizes the reasonable allocation and efficient utilization of resources, and avoids resource waste; and when the initial calculation state is met and the termination calculation condition is not met, the scheduling platform updates the state value at least once according to the set objective function and the system state information of each current scheduling state to find and determine the state optimal value of each scheduling state, so that the scheduling platform gradually approaches and achieves the optimal scheduling effect in continuous iteration and optimization, and improves the flexibility and robustness of task scheduling; in addition, when the task scheduling set is updated and / or the available resource set is updated, only the affected scheduling state part is updated and recalculated, so that the scheduling platform can respond in real time when resources and tasks change, and maintain efficient scheduling performance by optimizing the optimal value of state information.
[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.
[0013] Figure 1 is a flowchart of a task scheduling method shown in the first embodiment of the present disclosure;
[0014] Figure 2 is a flowchart of a task scheduling method shown in the second embodiment of the present disclosure;
[0015] Figure 3is a flowchart of a task scheduling method shown in the third embodiment of the present disclosure;
[0016] Figure 4 is a flowchart of a task scheduling method shown in the fourth embodiment of the present disclosure;
[0017] Figure 5 is a schematic diagram of the principle of the task scheduling method shown in the embodiment of the present disclosure;
[0018] Figure 6 is a structural diagram of a task scheduling device shown in the fifth embodiment of the present disclosure;
[0019] Figure 7 It is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0020] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.
[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0022] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.
[0023] In the related art, scheduling methods tend to make decisions by choosing the options that seem to be the best at the moment. This strategy may lead to suboptimal global results. It is also difficult to deal with complex constraints (such as resource limitations, time windows, dependencies, etc.) effectively. These constraints increase the complexity of the problem, making it impossible for simple scheduling methods to obtain feasible solutions or extremely inefficient. It is difficult to adjust scheduling strategies in a timely and flexible manner for dynamically changing environments and system states or uncertainties in different types of task requirements. For large-scale scheduling, the computational complexity of task scheduling using an exhaustive method is very high and difficult to obtain a solution within a reasonable time. When faced with large-scale tasks or multiple machines, as the scale of the problem increases, the computational complexity increases exponentially. In addition, the dependencies between tasks are complex and error-prone to handle, resulting in the inability to accurately and effectively schedule tasks.
[0024] Therefore, in order to solve the above problems, the present disclosure proposes a task scheduling method and device.
[0025] The task scheduling method and device according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.
[0026] Figure 1 1 is a flow chart of a task scheduling method according to the first embodiment of the present disclosure, wherein the task scheduling method according to the embodiment of the present disclosure is applied to a scheduling platform.
[0027] like Figure 1 As shown, the task scheduling method includes the following steps:
[0028] Step 101, obtaining a task scheduling set and an available resource set.
[0029] The task scheduling set includes at least one task to be scheduled and metadata information of at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource.
[0030] As an example, a task scheduling set includes at least one task to be scheduled and metadata information of at least one task to be scheduled. The task to be scheduled is a data task to be processed in the data asset platform (such as data conversion, calculation, etc.), and the metadata information of the task to be scheduled includes: execution time, priority, dependency, resource requirements, etc. Since at least one task to be scheduled and metadata information of at least one task to be scheduled are usually stored in a specific data storage, such as a database, message queue or file system, etc. Therefore, the scheduling platform obtains the task scheduling set through database query, message queue monitoring or file reading.
[0031] In addition, the available resource set includes: metadata information of at least one available resource, available resources include the number of central processing unit (CPU) cores, memory size, disk capacity, queue size, etc., and the metadata information of available resources includes: the number, type, capacity, available time, etc. of available resources. The scheduling platform obtains the available resource set through cluster monitoring, resource discovery mechanism, etc.
[0032] Step 102: Acquire the state space of the scheduling platform.
[0033] Among them, the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate the system state information of the scheduling platform under multiple orderly scheduling states.
[0034] In order to achieve load balancing of the scheduling platform and avoid waste of resources, as an example, scheduling planning is performed according to the task scheduling set and the available resource set to obtain the state space of the scheduling platform, wherein the state space is used to indicate the system state information of the scheduling platform under multiple ordered scheduling states. The multiple ordered scheduling states are the system states at each moment in the scheduling process. The system state information includes: the current time point, the state of each task to be scheduled, the current available quantity of each resource, the queue of tasks to be scheduled, the list of completed tasks, etc.
[0035] Step 103, evaluating the optimal state value of each scheduling state according to the system state information of each scheduling state in turn.
[0036] In order to facilitate the scheduling platform to optimize resource allocation and improve task scheduling efficiency, as a possible implementation method, by setting the initial calculation state and the termination calculation condition, the optimal state value of each scheduling state is calculated in turn. Among them, the initial calculation state is the starting condition for calculating the optimal state value of the scheduling state, and the termination calculation condition is the termination condition for calculating the optimal value of the scheduling state. The termination calculation conditions include: all tasks to be scheduled have completed scheduling, reached the set performance indicators, or the resource utilization rate reaches the threshold, etc.
[0037] Step 104: schedule at least one task to be scheduled based on the optimal state value of each scheduling state.
[0038] It should be understood that since the optimal state value usually represents the optimal scheduling effect of the task under the scheduling state, task scheduling based on the optimal state value of each scheduling state can ensure that the task is scheduled in the shortest time. For tasks to be scheduled with different priorities, scheduling based on the optimal state value of each scheduling state can achieve priority processing of high-priority tasks. Therefore, in order to better meet business needs and achieve load balancing and rational use of resources, as an example, based on the optimal state value of each scheduling state, a target scheduling scheme is determined, and the target scheduling scheme is adopted to schedule at least one task to be scheduled.
[0039] In order to improve the flexibility and adaptability of task scheduling, as a possible implementation method, the scheduling process is monitored in real time. When an abnormality is detected, the target scheduling plan is adjusted in time, and task scheduling is performed based on the adjusted target scheduling plan.
[0040] As an example, in the process of scheduling at least one task to be scheduled, at least one task to be scheduled and at least one available resource are monitored to obtain scheduling information of at least one task to be scheduled and resource information of at least one available resource; in response to abnormalities in the scheduling information and / or resource information, the target scheduling plan is adjusted to obtain an adjusted target scheduling plan; and the adjusted target scheduling plan is used to schedule an unscheduled task in at least one task to be scheduled.
[0041] That is to say, in the process of scheduling at least one task to be scheduled, the scheduling platform continuously monitors the status of at least one task to be scheduled and at least one available resource to obtain the scheduling information of the task and the resource information of the resource in real time; when the scheduling information or resource information is monitored to be abnormal, such as task execution delay, insufficient resource utilization or overload, etc., the scheduling platform responds in time and adjusts the target scheduling plan to generate an adjusted target scheduling plan, so that, based on the adjusted target scheduling plan, the unscheduled task in at least one task to be scheduled is scheduled. Therefore, the scheduling platform effectively handles various abnormal situations through timely monitoring and adjustment, avoids potential problems and bottlenecks, and ensures that the task can be efficiently and stably scheduled. In addition, the dynamic adjustment capability of the scheduling platform also helps to improve resource utilization, reduce resource waste, and thus reduce overall operating costs; at the same time, by continuously adjusting and optimizing the scheduling method, the scheduling platform can also continuously improve the performance and efficiency of scheduling to meet the needs of different application scenarios.
[0042] In order to further improve the performance and scheduling efficiency of the scheduling platform, as a possible implementation method, when the task scheduling set is updated and / or the available resource set is updated, only the affected scheduling state part is updated and recalculated.
[0043] As an example, in response to an update to a task scheduling set and / or an update to an available resource set, the state space is updated to obtain an updated state space; based on the state space and the updated state space, an updated scheduling state is determined; and based on the state information of the updated scheduling state, an optimal state value of the updated state information is evaluated.
[0044] That is to say, when the task scheduling set and / or the available resource set are updated, the scheduling platform reproduces the scheduling plan according to the updated task scheduling set and / or the available resource set to obtain the updated state space, and then, based on the comparison result between the initial state space and the updated state space, obtains the updated scheduling state, and then, according to the state information of these updated scheduling states, the scheduling platform re-evaluates and calculates to determine the optimal value of these updated state information; this process enables the scheduling platform to respond in real time when resources and tasks change, and maintains efficient scheduling performance by optimizing the optimal state value of the state information.
[0045] In summary, by obtaining a task scheduling set and an available resource set; obtaining a state space of a scheduling platform; obtaining a state space of a scheduling platform; evaluating the state optimal value of each scheduling state according to the system state information of each scheduling state in turn; scheduling at least one task to be scheduled based on the state optimal value of each scheduling state, thereby obtaining the state space of a scheduling platform through scheduling planning based on the task scheduling set and the available resource set, and evaluating the state optimal value of each scheduling state according to the system state information indicated by the state space of each scheduling state in turn, thereby scheduling at least one task to be scheduled based on the state optimal value of each scheduling state, thereby achieving load balancing and rational use of resources, and improving task execution efficiency, system performance and stability; the scheduling method based on the state optimal value makes the scheduling platform suitable for complex and changeable scheduling scenarios, and improves the flexibility and adaptability of the scheduling platform.
[0046] In order to clearly explain how to schedule at least one to-be-scheduled task based on the optimal state value of each scheduling state in the above embodiment, the present disclosure proposes another task scheduling method.
[0047] Figure 2 1 is a flow chart of a task scheduling method according to a second embodiment of the present disclosure. The task scheduling method according to the embodiment of the present disclosure is applied to a scheduling platform.
[0048] like Figure 2 As shown, the task scheduling method includes the following steps:
[0049] Step 201, obtaining a task scheduling set and an available resource set.
[0050] The task scheduling set includes at least one task to be scheduled and metadata information of the at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource.
[0051] Step 202: Acquire the state space of the scheduling platform.
[0052] Among them, the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate the system state information of the scheduling platform under multiple orderly scheduling states.
[0053] Step 203, evaluating the optimal state value of each scheduling state according to the system state information of each scheduling state in turn.
[0054] Step 204: determine the target subpath of each scheduling state based on the state optimal value of each scheduling state.
[0055] Each node in the target subpath is used to indicate a system status information, and the system status information includes the tasks to be scheduled and the available resources allocated to the node.
[0056] In order to schedule tasks at the best time and resource status, as a possible implementation method, according to the optimal state value of each scheduling state, the optimal decision path for switching the task scheduling state to the next scheduling state is solved to obtain the target subpath of any scheduling state.
[0057] It should be noted that, in order to enable the task to be executed in the best way under the optimal state value of a given scheduling state, each node in the target subpath is used to indicate a system state information, and the system state information includes the tasks to be scheduled and the available resources allocated to the node, that is, the target subpath of any scheduling state is used to indicate a series of decisions and operation steps that need to be followed to transition from the current scheduling state to the next scheduling state.
[0058] Step 205: Generate a target scheduling solution according to the target sub-paths of each scheduling state.
[0059] It should be understood that although the target sub-path provides a locally optimal decision path for each scheduling state, in order to make the entire task scheduling process proceed in the optimal way, as a possible implementation method, the information of all target sub-paths is combined to generate the final target scheduling solution.
[0060] As an example, a target path is determined based on the target sub-paths of each scheduling state; wherein the target path is used to indicate the best decision path between the first scheduling state and the last scheduling state in multiple ordered scheduling states; and a target scheduling plan is generated based on the target path.
[0061] That is to say, the target sub-paths of each scheduling state are combined to generate the optimal decision path of the entire scheduling process, that is, the target path, and then each scheduling state transition in the target path is converted into an actual scheduling instruction or operation. For example, if a scheduling state transition in the target path indicates that a task is assigned to a resource, the target scheduling plan contains this specific allocation operation, as well as the corresponding execution time, order, and other information.
[0062] It should be noted that since the target sub-path and target scheduling scheme are generated based on the real-time system status, the system can remain efficient and stable in different scenarios and meet various business needs.
[0063] Step 206: use the target scheduling solution to schedule at least one task to be scheduled.
[0064] It is important to understand that since the target scheduling plan includes the execution order of tasks, resource classification, and execution time, etc., the tasks to be scheduled are scheduled according to the generated target scheduling plan. The scheduling process includes steps such as allocating resources, starting tasks, and monitoring task execution.
[0065] It should be noted that the execution process of steps 201 to 203 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.
[0066] In summary, based on the optimal state value of each scheduling state, the target sub-path of each scheduling state is determined; according to the target sub-path of each scheduling state, a target scheduling plan is generated; and at least one task to be scheduled is scheduled using the target scheduling plan. Thus, by considering the optimal state value of each scheduling state, the target sub-path of each scheduling state is determined, so that the task obtains the required resources at the best time, thereby maximizing the overall performance of the system, reducing the waiting time and execution time of the task, and improving the system throughput and response speed. Based on the target sub-path of each scheduling state, a target scheduling plan is generated, which comprehensively considers system resources and task requirements, realizes the reasonable allocation and efficient utilization of resources, and avoids resource waste.
[0067] In order to clearly explain how the target sub-path of each scheduling state is determined based on the optimal state value of each scheduling state in the above embodiment, the present disclosure proposes another task scheduling method.
[0068] Figure 3 FIG. 1 is a flow chart of a task scheduling method according to a third embodiment of the present disclosure. The task scheduling method according to the embodiment of the present disclosure is applied to a scheduling platform.
[0069] like Figure 3 As shown, the task scheduling method includes the following steps:
[0070] Step 301, obtaining a task scheduling set and an available resource set.
[0071] The task scheduling set includes at least one task to be scheduled and metadata information of at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource.
[0072] Step 302: Acquire the state space of the scheduling platform.
[0073] Among them, the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate the system state information of the scheduling platform under multiple orderly scheduling states.
[0074] Step 303, evaluating the optimal state value of each scheduling state according to the system state information of each scheduling state in turn.
[0075] Step 304: Determine the scheduling sub-scheme to be adopted in each scheduling state according to the optimal state value of each scheduling state.
[0076] As an example, backtracking from the termination calculation condition, based on the calculated optimal state value of each scheduling state, determine the action to be taken in the corresponding scheduling state, that is, the scheduling sub-scheme adopted in each scheduling state. That is, adopting the corresponding scheduling sub-scheme in each scheduling state can achieve the optimal state value of each scheduling state. Among them, each scheduling sub-scheme includes: specific tasks to be executed, allocated resources, execution order of tasks, etc.
[0077] Step 305: construct an initial scheduling plan according to each scheduling sub-plan.
[0078] In order to achieve a comprehensive balance of various optimization objectives and thus achieve the global optimal effect, as a possible implementation method, the various scheduling sub-plans are integrated to form an initial scheduling plan.
[0079] As an example, a scheduling framework is obtained, which includes a unified interface and protocol so that each sub-scheme can communicate and collaborate with each other, and each scheduling sub-scheme is automatically integrated in the framework. Finally, through testing and verification, it is ensured that the initial scheduling scheme can meet actual needs, and necessary adjustments and optimizations are made.
[0080] Step 306 , based on the initial scheduling solution, solve the path for switching from any scheduling state to the next scheduling state to obtain the target sub-path of any scheduling state.
[0081] In order to optimize resource allocation and task execution order and maximize system performance, as a possible implementation method, based on the determined initial scheduling scheme, the transition path from any state in the scheduling process to its subsequent state is planned in detail, that is, for all possible transitions of any scheduling state, the optimal path for each transition is calculated, so as to determine the target sub-path from any scheduling state to the next scheduling state.
[0082] As an example, based on the initial scheduling plan, a path search is performed on the path from any scheduling state to the next scheduling state to obtain at least one candidate sub-path from any scheduling state to the next scheduling state; based on the task information and resource information on each candidate sub-path, the resource cost of each candidate sub-path is evaluated; based on the resource cost of each candidate sub-path, a target sub-path is determined from each candidate sub-path.
[0083] That is to say, based on the initial scheduling scheme, the path from any scheduling state to the next scheduling state is searched, and at least one candidate sub-path from any scheduling state to the next state can be obtained, wherein the candidate sub-path represents different task execution and resource allocation schemes. Then, based on the task information and resource information on each candidate sub-path, the resource cost required for each scheme is evaluated, including the resource usage, task execution time, etc. Finally, based on the evaluation results of these resource costs, the lowest cost or most efficient scheme is selected from the candidate sub-paths as the target sub-path to ensure that the scheduling process achieves the optimal or near-optimal effect in resource utilization.
[0084] Step 307: Generate a target scheduling solution according to the target sub-paths of each scheduling state.
[0085] Step 308: use the target scheduling scheme to schedule at least one task to be scheduled.
[0086] It should be noted that the execution process of steps 301 to 303 and steps 307 to 308 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.
[0087] In summary, according to the optimal state value of each scheduling state, the scheduling sub-scheme adopted in each scheduling state is determined; according to each scheduling sub-scheme, the initial scheduling scheme is constructed; based on the initial scheduling scheme, the path of switching from any scheduling state to the next scheduling state is solved to obtain the target sub-path of any scheduling state. Therefore, the scheduling sub-scheme under each scheduling state is determined according to the optimal state value of each scheduling state, which can ensure that the optimal resource allocation and task execution strategy are adopted in each scheduling state. The initial scheduling scheme constructed based on these sub-schemes has significantly improved overall efficiency and performance. Furthermore, based on the initial scheduling scheme, the path of switching from any scheduling state to the next state is solved to obtain the target sub-path, so that the scheduling process can be carried out along the optimal decision path at each step, which improves the efficiency and stability of the entire scheduling platform; at the same time, it realizes the maximum utilization of resources, reduces waste, and improves the efficiency of task scheduling.
[0088] In order to clearly explain how the above embodiment evaluates the optimal state value of each scheduling state according to the system state information of each scheduling state in turn, the present disclosure proposes another task scheduling method.
[0089] Figure 4 1 is a flow chart of a task scheduling method according to a fourth embodiment of the present disclosure. The task scheduling method according to the embodiment of the present disclosure is applied to a scheduling platform.
[0090] like Figure 4 As shown, the task scheduling method includes the following steps:
[0091] Step 401, obtaining a task scheduling set and an available resource set.
[0092] The task scheduling set includes at least one task to be scheduled and metadata information of at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource.
[0093] Step 402: Acquire the state space of the scheduling platform.
[0094] Among them, the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate the system state information of the scheduling platform under multiple orderly scheduling states.
[0095] Step 403, obtaining the initial calculation state and the calculation termination condition.
[0096] The calculation initial state is used to indicate that at least one task to be scheduled has not been executed and all available resources are available, and the calculation termination condition is used to indicate that at least one task to be scheduled has been executed or the set optimization goal has been achieved.
[0097] In order to make the task scheduling within a controllable range, as an example, the initial calculation state and the termination calculation condition are set in advance in the scheduling platform, so that the scheduling platform obtains the initial calculation state and the termination calculation condition.
[0098] The calculation initial state is used to indicate that at least one task to be scheduled has not been executed and all available resources are available, and the calculation termination condition is used to indicate that at least one task to be scheduled has been executed or the set optimization goal has been achieved.
[0099] Step 404, in response to satisfying the initial calculation state and not satisfying the termination calculation condition, based on the set objective function and according to the system state information of each scheduling state, perform at least one update process on the state value of each scheduling state to obtain the state optimal value of each scheduling state.
[0100] As an example, the initial calculation state is used as the starting point for calculating the optimal state value of each scheduling state, and the termination calculation condition is used as the end point for calculating the optimal state value of each scheduling state. When the initial calculation state is satisfied and the termination calculation condition is not met, the scheduling platform sequentially performs at least one state value update process for each scheduling state based on the current system state information according to the preset objective function (such as the cost function). That is, through iterative optimization, the state value of each scheduling state gradually approaches the optimal value, so that the entire scheduling process can achieve the goal of optimal performance or lowest cost under the premise of satisfying the constraints.
[0101] It should be noted that the i-th updating process in at least one updating process includes the following steps:
[0102] 1. Determine whether the calculation termination conditions are met;
[0103] 2. In response to the calculation termination condition being met, the update process is terminated, and the state value of each scheduling state obtained in the most recent update process is used as the state optimal value of each scheduling state;
[0104] 3. In response to the calculation termination condition not being met, based on the objective function, the state value obtained in the (i-1)th update process of each scheduling state is updated according to the system state information of each scheduling state.
[0105] Step 405: schedule at least one task to be scheduled based on the optimal state value of each scheduling state.
[0106] It should be noted that the execution process of steps 401 to 402 and step 405 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.
[0107] In summary, by obtaining the initial calculation state and the termination calculation condition; in response to the initial calculation state being met and the termination calculation condition not being met, the state value of each scheduling state is updated at least once based on the set objective function and the system state information of each scheduling state in turn to obtain the state optimal value of each scheduling state. Thus, when the initial calculation state is met and the termination calculation condition is not met, the scheduling platform updates the state value at least once based on the set objective function and the system state information of each current scheduling state to find and determine the state optimal value of each scheduling state, so that the scheduling platform gradually approaches and achieves the optimal scheduling effect in continuous iteration and optimization, which not only improves the scheduling efficiency, but also helps to reasonably allocate and fully utilize resources, thereby reducing costs, reducing waste, and optimizing overall performance. In addition, by continuously updating the state value, the system can better adapt to the dynamically changing environment and improve the flexibility and robustness of task scheduling.
[0108] Based on any embodiment of the present disclosure, Figure 5 As shown, the task scheduling method of the embodiment of the present disclosure can also be implemented based on the following steps:
[0109] Step 1. Data Collection and Definition
[0110] Get the set of tasks that need to be scheduled (i.e., task scheduling set), which includes all tasks that need to be scheduled and their attributes, such as execution time, priority, dependency, resource requirements, etc.; get the set of available scheduling resources and attributes (i.e., available resource set), which includes the number of central processing unit (CPU) cores, memory size, disk capacity, queue size, etc., and resource attributes include quantity, type, capacity, available time, etc.;
[0111] Step 2: Define the state space for scheduling
[0112] The state space is used to describe the system state at every moment in the scheduling process. The state space includes the current time point, the state of each task, the current available quantity of each resource, the queue of pending tasks, the list of completed tasks, etc., as well as the feasible operations, transition conditions and specified execution strategies between each state. The specified execution strategy means that a task can be started only when it is in the pending state and satisfies the dependency relationship, and the task will be paused or interrupted when the conditions are not met during the process.
[0113] Step 3: Set the initial state (initial calculation state) and termination conditions
[0114] The initial state is used to indicate that all tasks have not started and all resources are available; the termination calculation condition is used to indicate that all tasks have been completed or the preset optimization goal (such as the minimum total execution time or the minimum average task execution time) has been reached; for the initial state, a preprocessing step can be introduced to generate a topological sorting based on the task dependencies to facilitate subsequent scheduling. The termination calculation bar can be refined to check whether the status of all tasks is completed;
[0115] Step 4: Recursive calculation and optimization
[0116] Through recursive or iterative methods, the optimal value of the scheduling platform in each state is calculated step by step (for example, the average execution time of each task is short). The state table can be filled by using a top-down recursive method combined with memoization, or a bottom-up iterative method. The termination condition is recursively defined as a sub-problem, such as the completion of a single task, the cost of required resources, etc. The branch and bound algorithm pruning strategy can be used to reduce the search space, and the greedy algorithm can be used to simplify the calculation path to improve the overall recursive calculation efficiency;
[0117] Step 5: State migration detection
[0118] State transition detection is used to describe the change of the scheduling system (scheduling platform) from one state to another, and considers as many task allocation methods and resource utilization as possible, and ensures that the corresponding constraints are met. For example, it is set as follows: [f(t,\text{state})=\min{f(t+\Delta t,\text{new_state})+\text{cost}}], where (t) is the current time, (\text{state}) is the current state, (\Delta t) is the amount of time advancement, (\text{new_state}) is the potential new state, and (\text{cost}) is the cost of transferring from the current state to the new state. In each step, try to assign one or more tasks to the available resources and calculate the corresponding cost. The A* algorithm or greedy algorithm can be used to enhance the efficiency of state transfer. The optimal path can be quickly found by using the total time of the remaining unfinished tasks or the shortest estimated completion time of the tasks as heuristic functions, and the calculated states are stored in hash tables or dictionary tables to improve the retrieval of state information to avoid repeated calculations;
[0119] Step 6: Backtrack to obtain the scheduling plan
[0120] After the solution is completed, the optimal decision path of each state is recorded to obtain a specific scheduling plan; the optimal decision path can be obtained by searching the state space using depth-first or breadth-first, and then the optimal predecessor state of each state is recorded using the task predecessor node, so that the scheduling plan is reversely constructed after the optimal value is calculated to ensure that all decisions from the initial state to the terminal state are optimal;
[0121] Step 7: When a new task is added or the resource status changes, the task queue is updated and the priority and status of the current task are re-evaluated. The status table can be partially updated and the affected part can be recalculated without completely recalculating all the statuses.
[0122] Can dynamically adjust strategies and solution methods
[0123] Corresponding to the task scheduling method provided in the above-mentioned embodiment, the present disclosure also provides a task scheduling device. Since the task scheduling device provided in the embodiment of the present disclosure corresponds to the task scheduling method provided in the above-mentioned embodiment, the implementation method of the task scheduling method is also applicable to the task scheduling device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.
[0124] Figure 6 1 is a schematic diagram of the structure of a task scheduling device shown in the fifth embodiment of the present disclosure. The task scheduling device of the embodiment of the present disclosure is applied to a scheduling platform.
[0125] like Figure 6 As shown, the task scheduling device 600 includes: a first acquisition module 610 , a second acquisition module 620 , an evaluation module 630 and a scheduling module 640 .
[0126] Among them, the first acquisition module 610 is used to obtain a task scheduling set and an available resource set; wherein the task scheduling set includes at least one task to be scheduled and metadata information of at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource; the second acquisition module 620 is used to obtain the state space of the scheduling platform; wherein the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate the system state information of the scheduling platform under multiple ordered scheduling states; the evaluation module 630 is used to evaluate the state optimal value of each scheduling state according to the system state information of each scheduling state in turn; the scheduling module 640 is used to schedule at least one task to be scheduled based on the state optimal value of each scheduling state.
[0127] As a possible implementation method of the embodiment of the present disclosure, the scheduling module 640 is used to determine the target sub-path of each scheduling state based on the state optimal value of each scheduling state; wherein each node in the target sub-path is used to indicate a system state information, and the system state information includes the tasks to be scheduled and the available resources allocated to the node; according to the target sub-path of each scheduling state, a target scheduling scheme is generated; and at least one task to be scheduled is scheduled using the target scheduling scheme.
[0128] As a possible implementation method of the embodiment of the present disclosure, the scheduling module 640 is used to determine the target path according to the target sub-path of each scheduling state; wherein the target path is used to indicate the best decision path between the first scheduling state and the last scheduling state in the multiple ordered scheduling states; and generate the target scheduling plan according to the target path.
[0129] As a possible implementation method of the embodiment of the present disclosure, the scheduling module 640 is used to determine the scheduling sub-plan to be adopted in each scheduling state according to the state optimal value of each scheduling state; construct an initial scheduling plan according to each scheduling sub-plan; based on the initial scheduling plan, solve the path for switching from any scheduling state to the next scheduling state to obtain the target sub-path of any scheduling state.
[0130] As a possible implementation method of the embodiment of the present disclosure, the scheduling module 640 is used to perform a path search for the path of switching from any scheduling state to the next scheduling state based on the initial scheduling plan to obtain at least one candidate sub-path of switching from any scheduling state to the next scheduling state; evaluate the resource cost of each candidate sub-path according to the task information and resource information on each candidate sub-path; and determine the target sub-path from each candidate sub-path according to the resource cost of each candidate sub-path.
[0131] As a possible implementation of the embodiment of the present disclosure, the task scheduling device 600 further includes: an adjustment module.
[0132] Among them, the adjustment module is used to monitor at least one task to be scheduled and the at least one available resource during the process of scheduling at least one task to be scheduled, so as to obtain the scheduling information of at least one task to be scheduled and the resource information of at least one available resource; in response to the abnormality of the scheduling information and / or resource information, adjust the target scheduling plan to obtain an adjusted target scheduling plan; and use the adjusted target scheduling plan to schedule the unscheduled task in at least one task to be scheduled.
[0133] As a possible implementation method of the embodiment of the present disclosure, the evaluation module 630 is used to obtain an initial calculation state and a termination calculation condition; wherein the calculation initial state is used to indicate that at least one task to be scheduled has not been executed and all available resources are available, and the termination calculation condition is used to indicate that at least one task to be scheduled has been executed or the set optimization goal has been achieved; in response to satisfying the initial calculation state and not satisfying the termination calculation condition, based on the set objective function and according to the system state information of each scheduling state, the state value of each scheduling state is updated at least once to obtain the optimal state value of each scheduling state.
[0134] As a possible implementation method of an embodiment of the present disclosure, the i-th update process in at least one update process includes: determining whether a calculation termination condition is satisfied; in response to satisfying the calculation termination condition, ending the update process, and using the state value of each scheduling state obtained in the most recent update process as the optimal state value of each scheduling state; in response to not satisfying the calculation termination condition, based on the objective function, updating the state value obtained in the i-1-th update process of each scheduling state according to the system state information of each scheduling state.
[0135] As a possible implementation of the embodiment of the present disclosure, the task scheduling device 600 further includes: an update module.
[0136] Among them, the update module is used to update the state space in response to the update of the task scheduling set and / or the update of the available resource set to obtain an updated state space; determine the updated scheduling state based on the state space and the updated state space; and evaluate the state optimal value of the updated state information based on the state information of the updated scheduling state.
[0137] The task scheduling device of the embodiment of the present disclosure obtains a task scheduling set and an available resource set; obtains the state space of the scheduling platform; obtains the state space of the scheduling platform; evaluates the state optimal value of each scheduling state according to the system state information of each scheduling state in turn; and schedules at least one task to be scheduled based on the state optimal value of each scheduling state. Thus, scheduling planning is performed based on the task scheduling set and the available resource set to obtain the state space of the scheduling platform, and the system state information indicated by the state space of each scheduling state in turn evaluates the state optimal value of each scheduling state, thereby scheduling at least one task to be scheduled based on the state optimal value of each scheduling state, thereby achieving task scheduling in the shortest time, and giving priority to high-priority tasks, thereby better meeting business needs and achieving load balancing and rational use of resources.
[0138] In an exemplary embodiment, an electronic device is also provided.
[0139] Among them, electronic equipment includes:
[0140] processor;
[0141] a memory for storing processor-executable instructions;
[0142] The processor is configured to execute instructions to implement the task scheduling method proposed in any of the aforementioned embodiments.
[0143] As an example, Figure 7 is a schematic diagram of the structure of an electronic device 700 shown in an exemplary embodiment of the present disclosure, such as Figure 7 As shown, the electronic device 700 may further include:
[0144] The memory 710 and the processor 720, a bus 730 connecting different components (including the memory 710 and the processor 720), the memory 710 stores a computer program, and when the processor 720 executes the program, the task scheduling method described in the embodiment of the present disclosure is implemented.
[0145] Bus 730 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.
[0146] The electronic device 700 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 700, including volatile and non-volatile media, removable and non-removable media.
[0147] The memory 710 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 740 and / or cache memory 750. The server 700 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 760 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 730 via one or more data medium interfaces. The memory 710 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.
[0148] A program / utility 780 having a set (at least one) of program modules 770 may be stored, for example, in the memory 710, such program modules 770 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 770 generally perform the functions and / or methods of the embodiments described in the present disclosure.
[0149] The electronic device 700 may also communicate with one or more external devices 790 (e.g., keyboard, pointing device, display 791, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 792. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 793. As shown, the network adapter 793 communicates with other modules of the electronic device 700 through a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0150] The processor 720 executes various functional applications and data processing by running the programs stored in the memory 710 .
[0151] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the task scheduling method of the embodiment of the present disclosure, and will not be repeated here.
[0152] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to complete the task scheduling method proposed in any of the above embodiments. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0153] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the task scheduling method proposed in any of the above embodiments is implemented.
[0154] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0155] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A task scheduling method, It is characterized in that Applied to the dispatching platform, including: Acquire a task scheduling set and an available resource set; wherein the task scheduling set includes at least one task to be scheduled and metadata information of the at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource; Acquire the state space of the scheduling platform; wherein the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate system state information of the scheduling platform in multiple ordered scheduling states; According to the system status information of each scheduling state, evaluating the optimal state value of each scheduling state in turn; The at least one task to be scheduled is scheduled based on the optimal state value of each scheduling state.
2. The method according to claim 1, It is characterized in that The step of scheduling the at least one task to be scheduled based on the optimal state value of each scheduling state includes: Determine a target subpath for each of the scheduling states based on the state optimal value of each of the scheduling states; wherein each node in the target subpath is used to indicate a system state information, and the system state information includes the to-be-scheduled tasks and available resources allocated to the node; Generate a target scheduling solution according to the target subpath of each scheduling state; The at least one task to be scheduled is scheduled using the target scheduling scheme.
3. The method according to claim 2, It is characterized in that The generating a target scheduling solution according to the target sub-path of each scheduling state includes: Determine a target path according to the target subpaths of each of the scheduling states; wherein the target path is used to indicate the best decision path between the first scheduling state and the last scheduling state in the plurality of ordered scheduling states; The target scheduling solution is generated according to the target path.
4. The method according to claim 2, It is characterized in that The determining of the target subpath of each scheduling state based on the state optimal value of each scheduling state includes: Determining a scheduling sub-plan to be adopted in each of the scheduling states according to the optimal state value of each of the scheduling states; According to each of the scheduling sub-plans, construct an initial scheduling plan; Based on the initial scheduling scheme, a path for switching from any scheduling state to the next scheduling state is solved to obtain a target sub-path of any scheduling state.
5. The method according to claim 4, It is characterized in that The step of solving a path for switching from any scheduling state to a next scheduling state based on the initial scheduling scheme to obtain a target subpath of any scheduling state includes: Based on the initial scheduling scheme, a path search is performed on a path for switching from any scheduling state to a next scheduling state to obtain at least one candidate sub-path for switching from any scheduling state to the next scheduling state; Evaluate the resource cost of each candidate subpath according to the task information and resource information on each candidate subpath; The target sub-path is determined from the candidate sub-paths according to the resource costs of the candidate sub-paths.
6. The method according to claim 2, It is characterized in that The method further comprises: In the process of scheduling the at least one task to be scheduled, monitoring the at least one task to be scheduled and the at least one available resource to obtain scheduling information of the at least one task to be scheduled and resource information of the at least one available resource; In response to the scheduling information and / or the resource information being abnormal, adjusting the target scheduling scheme to obtain an adjusted target scheduling scheme; The adjusted target scheduling scheme is adopted to schedule the unscheduled tasks among the at least one task to be scheduled.
7. The method according to claim 1, It is characterized in that The step of evaluating the optimal state value of each scheduling state according to the system state information of each scheduling state in turn includes: Acquire an initial computing state and a termination computing condition; wherein the initial computing state is used to indicate that the at least one task to be scheduled has not been executed and the available resources are all available, and the termination computing condition is used to indicate that the at least one task to be scheduled has been executed or a set optimization goal has been reached; In response to satisfying the initial calculation state and not satisfying the termination calculation condition, based on the set objective function and according to the system state information of each scheduling state, at least one update process is performed on the state value of each scheduling state to obtain the optimal state value of each scheduling state.
8. The method according to claim 7, It is characterized in that The i-th updating process in the at least one updating process comprises: Determine whether the calculation termination condition is met; In response to the calculation termination condition being met, the update process is terminated, and the state value of each scheduling state obtained in the most recent update process is used as the state optimal value of each scheduling state; In response to the calculation termination condition not being met, based on the objective function and according to the system state information of each scheduling state, the state value obtained in the (i-1)th updating process of each scheduling state is updated.
9. The method according to claim 1, It is characterized in that The method further comprises: In response to an update of the task scheduling set and / or an update of the available resource set, updating the state space to obtain an updated state space; Determining an updated scheduling state according to the state space and the updated state space; According to the state information of the updated scheduling state, an optimal state value of the updated state information is evaluated.
10. A task scheduling device, It is characterized in that Applied to the dispatching platform, including: A first acquisition module is used to acquire a task scheduling set and an available resource set; wherein the task scheduling set includes at least one task to be scheduled and metadata information of the at least one task to be scheduled, and the available resource set includes metadata information of at least one available resource; A second acquisition module is used to acquire the state space of the scheduling platform; wherein the state space is obtained by scheduling planning based on the task scheduling set and the available resource set, and is used to indicate system state information of the scheduling platform in multiple ordered scheduling states; An evaluation module, used for evaluating the optimal state value of each scheduling state according to the system state information of each scheduling state in turn; The scheduling module is used to schedule the at least one task to be scheduled based on the optimal state value of each scheduling state.
Citation Information
Cited By
Information collection method and device in penetration test, computer equipment and medium
CN120832677A
Information collection method, device, computer equipment and medium in penetration testing
CN120832677B
