Resource scheduling method and device, equipment and medium
Through real-time monitoring and dynamic adjustment of the number of executors, the problem of unreasonable resource scheduling of business queues and resource pools in chip test simulation is solved, which improves task processing efficiency and reduces management costs.
Patent Information
- Application Number
- CN202411124485.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-08
Smart Images

Figure CN118916176A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of chip test simulation technology, and in particular to a resource scheduling method, device, equipment and medium. Background Art
[0002] With the rapid development of integrated circuit technology, the complexity of chip design continues to increase. Chip simulation testing has become an important part of ensuring the correctness of chip design. As the complexity of integrated circuits increases, the computing resources and time required for simulation verification also increase. In the simulation verification process, task scheduling and resource management become particularly important because they directly affect the efficiency and success rate of simulation. OpenLava is an open source workload scheduler based on IBM Spectrum LSF (LoadSharing Facility), designed for high-performance computing (HPC) and analysis applications. In the field of chip simulation verification, OpenLava's efficient scheduling capabilities can help design teams better manage simulation task queues and optimize the use of computing resources, thereby accelerating the simulation process and reducing costs.
[0003] Although OpenLava has significant advantages in resource scheduling, in the field of chip testing and simulation, the resource scheduling solution based on OpenLava is static, that is, users pre-configure the execution machines of each business queue, and then use the execution machines to process tasks. This leads to unreasonable resource scheduling between business queues and resource pools, which affects the processing efficiency of tasks.
[0004] It can be seen that how to ensure reasonable resource scheduling between the service queue and the resource pool to ensure task processing efficiency is a problem that technical personnel in this field need to solve. Summary of the invention
[0005] The purpose of the embodiments of the present invention is to provide a resource scheduling method, device, equipment and medium, which can ensure reasonable resource scheduling between the service queue and the resource pool to ensure task processing efficiency. The specific scheme is as follows:
[0006] In a first aspect, the present invention discloses a resource scheduling method, which is applied to a workload scheduler, comprising:
[0007] Determine a first execution machine state of a current service queue and a second execution machine state in a dynamic resource pool, and determine whether a preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state;
[0008] If the preset resource mismatch condition is met, task information of the chip test task in the current service queue is obtained, and weighted summation of the task information is performed to generate a comprehensive load index of the current service queue;
[0009] Determine a magnitude relationship between the comprehensive load index and a current index threshold of the current service queue, and generate an execution machine usage evaluation result based on the magnitude relationship;
[0010] A target resource scheduling strategy is determined according to the evaluation result of the use of the execution machine and the state of the second execution machine, and the number of execution machines in the current service queue is adjusted according to the target resource scheduling strategy.
[0011] Optionally, the determining whether a preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state includes:
[0012] If the first execution machine state of the current business queue indicates that the number of execution machines in the current business queue that are in an idle state is less than a first preset threshold and the second execution machine state in the dynamic resource pool indicates that the number of execution machines in the dynamic resource pool that are in an idle state is greater than a second preset threshold, it is determined that the first preset resource mismatch condition is currently met;
[0013] If the first execution machine state of the current business queue indicates that the number of execution machines in the current business queue that are in an idle state is greater than the third preset threshold and the second execution machine state in the dynamic resource pool indicates that the number of execution machines in the dynamic resource pool that are in an idle state is less than the fourth preset threshold, it is determined that the second preset resource mismatch condition is currently met;
[0014] Correspondingly, the step of obtaining the task information of the chip test task in the current service queue includes:
[0015] If the first preset resource mismatch condition is currently met, the target chip test task in the current business queue is suspended, and the task information of all chip test tasks in the current business queue is obtained; wherein the target chip test task is a task to which an execution machine is not assigned.
[0016] Optionally, the acquiring task information of the chip test task in the current service queue includes:
[0017] Collecting the number of tasks of the current service queue and the maximum queue capacity of the current service queue, and determining the task queue length index of the chip test task based on the ratio between the number of tasks and the maximum queue capacity;
[0018] The average task execution time of the current service queue is collected, and the task execution time index of the chip test task is determined based on the ratio between the average task execution time and a preset benchmark execution time.
[0019] Optionally, the weighted summing of the task information to generate a comprehensive load index of the current service queue includes:
[0020] Determine a first weight coefficient of the task queue length index and a second weight coefficient of the task execution time index;
[0021] Performing a weighted summation of the task queue length index and the task execution time index based on the first weight coefficient and the second weight coefficient to generate a comprehensive load index of the current service queue;
[0022] Accordingly, determining the magnitude relationship between the comprehensive load index and the current index threshold of the current service queue, and generating an execution machine usage evaluation result based on the magnitude relationship, includes:
[0023] Determine a current index threshold of the current service queue according to historical task execution data of the current service queue and system performance requirements;
[0024] If the comprehensive load index is less than the current index threshold, generating an executor usage evaluation result indicating that the current service queue needs to increase the number of executors;
[0025] If the comprehensive load index is not less than the current index threshold, an executor usage evaluation result is generated, indicating that the current service queue needs to reduce the number of executors.
[0026] Optionally, determining a target resource scheduling strategy according to the evaluation result of the use of the execution machine and the state of the second execution machine, and adjusting the number of execution machines of the current service queue according to the target resource scheduling strategy includes:
[0027] If the evaluation result of the execution machine usage indicates that the current business queue needs to increase the number of execution machines and the second execution machine state indicates that the number of execution machines in the dynamic resource pool that are idle is greater than the second preset threshold, then the target resource scheduling strategy is determined to be the target resource capacity expansion strategy, and the execution machines in the dynamic resource pool are included in the current business queue according to the target resource capacity expansion strategy to increase the number of execution machines in the current business queue;
[0028] If the evaluation result of the execution machine usage indicates that the current business queue needs to reduce the number of execution machines and the second execution machine state indicates that the number of execution machines in the dynamic resource pool is less than the fourth preset threshold, then the target resource scheduling strategy is determined to be the target resource capacity reduction strategy, and the idle execution machines in the current business queue are included in the dynamic resource pool according to the target resource capacity reduction strategy to reduce the number of execution machines in the current business queue.
[0029] Optionally, the resource scheduling method further includes:
[0030] Get the current operation instruction based on the preset interface;
[0031] If the current operation instruction is a viewing instruction, the target display information corresponding to the current operation instruction is filtered out from the current status and configuration information of each execution machine in the current business queue, the current status and configuration information of each execution machine in the dynamic resource pool, and the target resource scheduling strategy, and the target display information is displayed on a preset display interface;
[0032] If the current operation instruction is a resource adjustment instruction, the execution machines in the current service queue and the execution machines in the dynamic resource pool are adjusted according to the current operation instruction.
[0033] Optionally, determining a target resource scheduling strategy according to the evaluation result of the execution machine usage and the state of the second execution machine includes:
[0034] Determining the priority of each chip test task in the current service queue;
[0035] A target resource scheduling strategy is determined according to the execution machine usage evaluation result, the second execution machine state, and the priority of each of the chip test tasks.
[0036] In a second aspect, the present invention discloses a resource scheduling device, which is applied to a workload scheduler, comprising:
[0037] A resource judgment module, used to determine the first execution machine state of the current service queue and the second execution machine state in the dynamic resource pool, and determine whether a preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state;
[0038] An index generating module, configured to obtain task information of chip test tasks in the current service queue if the preset resource mismatch condition is met, and weighted sum the task information to generate a comprehensive load index of the current service queue;
[0039] An execution machine evaluation module, used to determine the size relationship between the comprehensive load index and the current index threshold of the current service queue, and generate an execution machine usage evaluation result based on the size relationship;
[0040] A resource scheduling module is used to determine a target resource scheduling strategy according to the evaluation result of the use of the execution machine and the state of the second execution machine, and adjust the number of execution machines in the current business queue according to the target resource scheduling strategy.
[0041] In a third aspect, the present invention discloses a resource scheduling device, comprising:
[0042] Memory for storing computer programs;
[0043] The processor is used to execute the computer program to implement the steps of the resource scheduling method disclosed above.
[0044] In a fourth aspect, the present invention discloses a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the resource scheduling method disclosed above are implemented.
[0045] It can be seen that the present invention is applied to a workload scheduler to determine the first execution machine state of the current business queue and the second execution machine state in the dynamic resource pool, and determine whether a preset resource mismatch condition is currently met based on the first execution machine state and the second execution machine state; if the preset resource mismatch condition is met, the task information of the chip test task in the current business queue is obtained, and the task information is weighted and summed to generate a comprehensive load index of the current business queue; determine the size relationship between the comprehensive load index and the current index threshold of the current business queue, and generate an execution machine usage evaluation result based on the size relationship; determine the target resource scheduling strategy based on the execution machine usage evaluation result and the second execution machine state, and adjust the number of execution machines in the current business queue according to the target resource scheduling strategy.
[0046] Beneficial effect: When a resource mismatch occurs between a business queue and a resource pool, the workload scheduler (i.e., OpenLava) of the present invention generates an executor usage evaluation result of the current business queue based on the task information of the chip test task in the current business queue, and adjusts the number of executors in the current business queue according to the target resource scheduling strategy determined according to the executor usage evaluation result and the state of the second executor. That is to say, the present invention monitors the business queue in real time, and when a resource mismatch occurs, intelligently generates a target resource scheduling strategy, and automatically schedules the resources of the business queue to ensure reasonable resource scheduling between the business queue and the resource pool, thereby ensuring task processing efficiency, and eliminating the need for manual resource configuration, thereby reducing management costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0048] Figure 1 A flow chart of a resource scheduling method provided by an embodiment of the present invention;
[0049] Figure 2 A specific resource scheduling method flow chart provided by an embodiment of the present invention;
[0050] Figure 3 A specific resource reduction flow chart provided by an embodiment of the present invention;
[0051] Figure 4 A schematic diagram of the structure of a resource scheduling device provided by an embodiment of the present invention;
[0052] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.
[0055] OpenLava is an open source workload scheduler based on IBM Spectrum LSF, designed for high-performance computing and analytical applications. In the field of chip simulation and verification, OpenLava's efficient scheduling capabilities can help design teams better manage simulation task queues and optimize the use of computing resources, thereby accelerating the simulation process and reducing costs.
[0056] Although OpenLava has significant advantages in resource scheduling, in the field of chip testing and simulation, the existing OpenLava platform adopts a static resource allocation strategy and cannot dynamically adjust the number of executors according to the actual needs of the task. When the number of tasks in a queue suddenly increases, since the resources have been statically allocated, the queue may be short of resources, while the resources of other queues are idle, resulting in resource waste. The static resource scheduling solution requires administrators to manually allocate and adjust resources, which is not only labor-intensive but also prone to errors. At the same time, as the cluster scale expands, management costs will further increase.
[0057] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0058] Next, a resource scheduling provided by an embodiment of the present invention is introduced in detail. Figure 1 A resource scheduling method provided in an embodiment of the present invention includes:
[0059] Step S11: Determine the first execution machine state of the current service queue and the second execution machine state in the dynamic resource pool, and determine whether a preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state.
[0060] In this embodiment, the determination of whether the preset resource mismatch condition is currently met based on the first execution machine state and the second execution machine state includes: if the first execution machine state of the current business queue indicates that the number of execution machines in the current business queue that are in an idle state is less than a first preset threshold and the second execution machine state in the dynamic resource pool indicates that the number of execution machines in the dynamic resource pool that are in an idle state is greater than a second preset threshold, then it is determined that the first preset resource mismatch condition is currently met; if the first execution machine state of the current business queue indicates that the number of execution machines in the current business queue that are in an idle state is greater than a third preset threshold and the second execution machine state in the dynamic resource pool indicates that the number of execution machines in the dynamic resource pool that are in an idle state is less than a fourth preset threshold, then it is determined that the second preset resource mismatch condition is currently met. Monitor the status of the first executor in the current business queue and the status of the second executor in the dynamic resource pool in real time, that is, monitor the number of busy executors and the number of idle executors in the current business queue in real time, and monitor the number of busy executors and the number of idle executors in the dynamic resource pool in real time. If the number of idle executors in the current business queue is too small, and the number of idle executors in the dynamic resource pool is too large, it is necessary to re-schedule resources, that is, to meet the first preset resource mismatch condition. For example, set the first preset threshold to 1 and the second preset threshold to 5. If the number of idle executors in the current business queue is 0 and the number of idle executors in the dynamic resource pool is 6, then determine the current The first preset resource mismatch condition is met; if the number of idle executors in the current business queue is too large, and the number of idle executors in the dynamic resource pool is too small, it is necessary to re-schedule resources, that is, to meet the second preset resource mismatch condition. For example, the third preset threshold is set to 3, and the fourth preset threshold is set to 4. If the number of idle executors in the current business queue is 5, and the number of idle executors in the dynamic resource pool is 2, it is determined that the second preset resource mismatch condition is currently met; regardless of whether the first preset resource mismatch condition or the second preset resource mismatch condition is currently met, it means that the number of executors in the current business queue is not reasonably allocated, that is, resource scheduling is needed to try to meet the current business queue's demand for executors.
[0061] It can be understood that if the first execution machine state of the current business queue represents that the number of execution machines in the idle state in the current business queue is less than the first preset threshold and the second execution machine state in the dynamic resource pool represents that the number of execution machines in the idle state in the dynamic resource pool is 0, it means that both the number of execution machines in the current business queue and the number of execution machines in the dynamic resource pool are relatively small, so it is impossible to reallocate execution machines from the dynamic resource pool to the current business queue at this time. The current business queue can suspend the chip test task of the unallocated execution machine, and monitor the first execution machine state and the second execution machine state in real time so as to While waiting for an idle executor in the current business queue or an idle executor in the dynamic resource pool, use the executor to process the suspended chip test task; if the first executor state of the current business queue indicates that the number of idle executors in the current business queue is greater than the third preset threshold and the second executor state in the dynamic resource pool indicates that the number of idle executors in the dynamic resource pool is greater than the second preset threshold, it means that the number of executors in the current business queue and the number of executors in the dynamic resource pool are relatively large, and resource scheduling may not be performed at this time to reduce the computing power required for resource scheduling.
[0062] Step S12: If the preset resource mismatch condition is met, the task information of the chip test task in the current service queue is obtained, and the task information is weighted and summed to generate a comprehensive load index of the current service queue.
[0063] In this embodiment, the task information of the chip test task in the current service queue is obtained, including: if the first preset resource mismatch condition is currently met, the target chip test task in the current service queue is suspended, and the task information of all chip test tasks in the current service queue is obtained; wherein the target chip test task is a task that is not assigned an execution machine. It can be understood that if the first preset resource mismatch condition is currently met, it means that there is no idle execution machine in the current service queue, then the chip test task that is not assigned an execution machine needs to be suspended first, and wait until there is an idle execution machine in the subsequent current service queue, and then process the target chip test task.
[0064] In this embodiment, the task information of the chip test task in the current business queue is obtained, including: collecting the number of tasks in the current business queue and the maximum queue capacity of the current business queue, and determining the task queue length index of the chip test task based on the ratio between the number of tasks and the maximum queue capacity; collecting the average task execution time of the current business queue, and determining the task execution time index of the chip test task based on the ratio between the average task execution time and a preset benchmark execution time. The task information of the chip test task in the current business queue is collected. The task information may specifically be the number of tasks in the current business queue, the maximum queue capacity of the current business queue, and the average task execution time of the current business queue. Next, the task queue length index (TQI) and the task execution time index (TEI) are calculated. The task queue length index (TQI) of the chip test task can reflect the task backlog situation, and the task execution time index (TEI) can reflect the task execution efficiency.
[0065] The specific formula is as follows:
[0066] TQI=(number of tasks in the current business queue / maximum queue capacity of the current business queue)*100%;
[0067] TEI = average task execution time / preset benchmark execution time.
[0068] In this embodiment, the weighted summation of the task information to generate the comprehensive load index of the current service queue includes: determining a first weight coefficient of the task queue length index and a second weight coefficient of the task execution time index; weighted summing the task queue length index and the task execution time index based on the first weight coefficient and the second weight coefficient to generate the comprehensive load index of the current service queue. The first weight coefficient (α) of the task queue length index and the second weight coefficient (1-α) of the task execution time index are determined according to the system policy and current conditions. It can be understood that α (0≤α≤1) is used to balance the impact between TQI and TEI. The formula of the comprehensive load index CLI of the current service queue is as follows:
[0069] CLI=α*TQI+(1-α)*TEI.
[0070] Step S13: Determine the size relationship between the comprehensive load index and the current index threshold of the current service queue, and generate an execution machine usage evaluation result based on the size relationship.
[0071] In this embodiment, the determination of the size relationship between the comprehensive load index and the current index threshold of the current business queue, and generating an execution machine usage evaluation result based on the size relationship, includes: determining the current index threshold of the current business queue according to the historical task execution data of the current business queue and the system performance requirements; if the comprehensive load index is less than the current index threshold, generating an execution machine usage evaluation result indicating that the current business queue needs to increase the number of execution machines; if the comprehensive load index is not less than the current index threshold, generating an execution machine usage evaluation result indicating that the current business queue needs to reduce the number of execution machines. It should be noted that, in this embodiment, the current index threshold of the current business queue changes dynamically, that is, the corresponding index threshold in different situations is also different. The current index threshold of the current business queue is determined based on historical data and system performance requirements. The historical data can be the historical task execution data of the current business queue. After determining the current index threshold, if the comprehensive load index is less than the current index threshold, it means that the number of executors in the current business queue is small and needs to be expanded. Therefore, an executor usage evaluation result is generated, which indicates that the number of executors in the current business queue needs to be increased. If the comprehensive load index is not less than the current index threshold, it means that the number of executors in the current business queue is large and needs to be reduced. Therefore, an executor usage evaluation result is generated, which indicates that the number of executors in the current business queue needs to be reduced.
[0072] Step S14: determining a target resource scheduling strategy according to the execution machine usage evaluation result and the second execution machine state, and adjusting the number of execution machines in the current service queue according to the target resource scheduling strategy.
[0073] In this embodiment, the target resource scheduling strategy is determined according to the evaluation result of the use of the execution machine and the state of the second execution machine, and the number of execution machines in the current business queue is adjusted according to the target resource scheduling strategy, including: if the evaluation result of the use of the execution machine indicates that the current business queue needs to increase the number of execution machines and the state of the second execution machine indicates that the number of execution machines in the idle state in the dynamic resource pool is greater than the second preset threshold, then the target resource scheduling strategy is determined to be a target resource capacity expansion strategy, and the execution machines in the dynamic resource pool are included in the current business queue according to the target resource capacity expansion strategy to increase the number of execution machines in the current business queue; if the evaluation result of the use of the execution machine indicates that the current business queue needs to reduce the number of execution machines and the state of the second execution machine indicates that the number of execution machines in the idle state in the dynamic resource pool is less than the fourth preset threshold, then the target resource scheduling strategy is determined to be a target resource capacity reduction strategy, and the execution machines in the idle state in the current business queue are included in the dynamic resource pool according to the target resource capacity reduction strategy to reduce the number of execution machines in the current business queue.
[0074] If the executor usage evaluation result indicates that the current business queue needs to increase the number of executors, it is necessary to check whether there are idle executors in the dynamic resource pool and the number of idle executors. That is to say, if the number of idle executors in the dynamic resource pool cannot meet the number of executors that the current business queue needs to increase, it is necessary to redetermine the final number of executors that need to be added to the current business queue, that is, determine the target resource scheduling strategy. Therefore, only when the executor usage evaluation result indicates that the current business queue needs to increase the number of executors and the second executor state indicates that the number of idle executors in the dynamic resource pool is greater than the second preset threshold, the target resource scheduling strategy is determined to be the target resource expansion strategy; wherein, the executor usage evaluation result includes the need to increase the number of executors, and the target resource expansion strategy includes information such as the executor identifier and the number of executors allocated to the current business queue in the dynamic resource pool. For example, if the executor usage evaluation result indicates that the current business queue needs to increase If the number of executors and the number of increased executors are 5, and the second executor state represents that the number of idle executors in the dynamic resource pool is greater than the second preset threshold value of 5, then the target resource scheduling strategy is determined to be the target resource expansion strategy, that is, 5 idle executors can be allocated from the dynamic resource pool to the current business queue, and the identifiers of these 5 idle executors are determined, and then the target resource expansion strategy incorporates the executors in the dynamic resource pool into the current business queue to increase the number of executors in the current business queue; for example, if the executor usage evaluation result represents that the current business queue needs to increase the number of executors and the number of increased executors is 10, and the second executor state represents that the number of idle executors in the dynamic resource pool is 7, which is greater than the second preset threshold value of 5, then the target resource scheduling strategy is determined to be the target resource expansion strategy, but since the number of idle executors in the dynamic resource pool is 7, the target resource expansion strategy represents that only these 7 idle executors can be incorporated into the current business queue.
[0075] If the execution machine usage evaluation result indicates that the current business queue needs to reduce the number of execution machines, it is necessary to check whether there is a need to add execution machines in the dynamic resource pool. That is to say, if the second execution machine state indicates that the number of execution machines in the idle state in the dynamic resource pool is less than the fourth preset threshold, it indicates that there is a need to add execution machines in the dynamic resource pool. Therefore, the target resource scheduling strategy is determined to be the target resource reduction strategy. Furthermore, the execution machine usage evaluation result may include the number of execution machines that need to be reduced and the execution machine identifier. The target resource reduction strategy includes the number of execution machines and the execution machine identifier included in the dynamic resource pool from the execution machines in the current business queue. For example, the execution machine usage evaluation result indicates that 6 idle execution machines in the current business queue can be included in the dynamic resource pool. The number of idle execution machines in the dynamic resource pool is less than the fourth preset threshold. Therefore, it is necessary to include these 6 idle execution machines in the dynamic resource pool to reduce the number of execution machines in the current business queue.
[0076] In this embodiment, it also includes: obtaining the current operation instruction based on a preset interface; if the current operation instruction is a viewing instruction, filtering out the target display information corresponding to the current operation instruction from the current status and configuration information of each executor in the current business queue, the current status and configuration information of each executor in the dynamic resource pool, and the target resource scheduling strategy, and displaying the target display information on the preset display interface; if the current operation instruction is a resource adjustment instruction, adjusting the executors in the current business queue and the executors in the dynamic resource pool according to the current operation instruction.
[0077] It should be noted that the workload scheduler also includes a preset interface and a preset display interface. The user issues different current operation instructions through the preset interface. The preset display interface can display the detailed configuration and current status of each executor in each business queue and the detailed configuration and current status of each executor in the dynamic resource pool. It can also be used to display information such as the current processing status of each chip test task. The user understands various information in the preset display interface and issues various operation instructions through the preset interface. For example, if the current operation instruction issued by the user is a viewing instruction, the target display information corresponding to the viewing instruction is filtered out from the various information that can be displayed, and then the target display information is displayed on the preset display interface. If the current operation instruction issued by the user is a resource adjustment instruction, the executor in the current business queue and the executor in the dynamic resource pool are adjusted according to the current operation instruction. Specifically, it can be to add, delete, and modify the executor in the current business queue and the dynamic resource pool. In this way, user-defined resource scheduling and user information viewing functions can be realized, which is more flexible.
[0078] Furthermore, there may be a variety of ways for users to issue operation instructions through a preset interface. For example, if the preset interface is a voice acquisition interface, then the user can issue the current operation instruction in the form of voice. The preset interface may also be a text input box, and the user enters the current operation instruction in the text input box. When the information of each node is displayed on the preset display interface, the preset interface may be displayed in the form of a button at the corresponding position of each node information, and the user can issue the corresponding operation instruction by touching each button. The issuance and acquisition of the current operation instruction are realized in a variety of forms to improve the user experience.
[0079] It can be seen that the present invention is applied to a workload scheduler to determine the first execution machine state of the current business queue and the second execution machine state in the dynamic resource pool, and determine whether a preset resource mismatch condition is currently met based on the first execution machine state and the second execution machine state; if the preset resource mismatch condition is met, the task information of the chip test task in the current business queue is obtained, and the task information is weighted and summed to generate a comprehensive load index of the current business queue; determine the size relationship between the comprehensive load index and the current index threshold of the current business queue, and generate an execution machine usage evaluation result based on the size relationship; determine the target resource scheduling strategy based on the execution machine usage evaluation result and the second execution machine state, and adjust the number of execution machines in the current business queue according to the target resource scheduling strategy.
[0080] Beneficial effect: When a resource mismatch occurs between a business queue and a resource pool, the workload scheduler (i.e., OpenLava) of the present invention generates an executor usage evaluation result of the current business queue based on the task information of the chip test task in the current business queue, and adjusts the number of executors in the current business queue according to the target resource scheduling strategy determined according to the executor usage evaluation result and the state of the second executor. That is to say, the present invention monitors the business queue in real time, and when a resource mismatch occurs, intelligently generates a target resource scheduling strategy, and automatically schedules the resources of the business queue to ensure reasonable resource scheduling between the business queue and the resource pool, thereby ensuring task processing efficiency, and eliminating the need for manual resource configuration, thereby reducing management costs.
[0081] See also Figure 2 The embodiment of the present invention discloses a specific resource scheduling method, which is applied to a workload scheduler. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, it includes:
[0082] Step S21: Determine the first execution machine state of the current service queue and the second execution machine state in the dynamic resource pool, and determine whether a preset resource mismatch condition is currently met based on the first execution machine state and the second execution machine state.
[0083] The workload scheduler (ie, OpenLava) includes a state monitoring module for monitoring the state of the first executor of the current service queue and the state of the second executor in the dynamic resource pool in real time to determine whether a preset resource mismatch condition is currently met.
[0084] Step S22: If the preset resource mismatch condition is met, the task information of the chip test task in the current service queue is obtained, and the task information is weighted and summed to generate a comprehensive load index of the current service queue.
[0085] The workload scheduler includes a data collection module, which uses the API (Application Programming Interface) interface provided by the workload scheduler to collect the number of tasks in the current business queue, the maximum queue capacity of the current business queue, and the average task execution time of the current business queue, so as to determine the task queue length index of the chip test task based on the ratio between the number of tasks and the maximum queue capacity, and determine the task execution time index of the chip test task based on the ratio between the average task execution time and the preset benchmark execution time.
[0086] The user submits the written chip test task to the OpenLava cluster, namely the workload scheduler, through the bsub command. The workload scheduler then assigns the chip test task to the corresponding business queue, so that each business queue uses the idle execution machine to process each chip test task.
[0087] Step S23: Determine the size relationship between the comprehensive load index and the current index threshold of the current service queue, and generate an execution machine usage evaluation result based on the size relationship.
[0088] The workload scheduler includes an evaluation module, which determines the size relationship between the comprehensive load index and the current index threshold of the current service queue, and generates an execution machine usage evaluation result based on the size relationship.
[0089] Step S24: Determine the priority of each chip test task in the current business queue; determine the target resource scheduling strategy according to the execution machine usage evaluation result, the second execution machine status and the priority of each chip test task, and adjust the number of execution machines in the current business queue according to the target resource scheduling strategy.
[0090] In the process of determining the target resource scheduling strategy, the priority of each chip test task in the current service queue, that is, the urgency of each task, may also be considered to ensure that tasks with higher priority levels are processed in a timely manner as much as possible.
[0091] The workload scheduler includes a policy decision module for generating a target resource scheduling policy; the workload scheduler includes a resource pool management module for scheduling each executor in the dynamic resource pool according to the target resource scheduling policy to adjust the number of executors in the current business queue to complete resource scheduling; further, the workload scheduler may also include a display module and a custom module, the display module displays various information, and the custom module adjusts relevant configurations and relevant policies according to the current operation instructions issued by the user.
[0092] It should be noted that in order to avoid adverse effects on the tasks being executed, in this embodiment, when adjusting the execution machine, only the execution machine in the idle state can be adjusted to achieve smooth expansion and reduction of the task queue; after completing the resource scheduling, the idle execution machine can be used to process the corresponding chip test tasks, and the task processing results can be saved in the specified directory so that the user can view the corresponding task processing results in the directory.
[0093] It can be seen that the present invention can monitor the resources of the business queue in real time. When resource scheduling is required, the execution machine of the business queue can be automatically and dynamically adjusted to ensure reasonable resource scheduling between the business queue and the resource pool; further, when determining the target resource scheduling strategy, the priority of each chip test task in the business queue is also considered to achieve a reasonable processing order for the tasks, and the smooth expansion and contraction of the business queue does not affect the execution of key tasks, thereby ensuring system stability.
[0094] Below Figure 3 The present invention is described by taking a specific resource scaling flow chart as an example. The user writes a task script and submits the task to the workload scheduler. The workload scheduler adds the task to the specified service queue and then monitors the current service queue in real time to schedule resources for the current service queue. The specific process is as follows:
[0095] 1) Monitor whether the current business queue has sufficient resources. If not, suspend the tasks of the unassigned execution machines. The specific process is to determine the first execution machine state of the current business queue and the second execution machine state in the dynamic resource pool. If the first execution machine state of the current business queue indicates that the number of idle execution machines in the current business queue is less than the first preset threshold and the second execution machine state in the dynamic resource pool indicates that the number of idle execution machines in the dynamic resource pool is greater than the second preset threshold, it is determined that the first preset resource mismatch condition is currently met, that is, the resources of the current business queue are insufficient, and suspend the tasks of the unassigned execution machines;
[0096] 2) Obtaining task information of chip test tasks in the current business queue; weighted summing the task information to generate a comprehensive load index of the current business queue; determining the size relationship between the comprehensive load index and the current index threshold of the current business queue, and generating an execution machine usage evaluation result based on the size relationship;
[0097] 3) Determine the target resource scheduling strategy based on the executor usage evaluation result and the second executor state. It can be understood that if the second executor state indicates that there are idle executor resource scheduling in the dynamic resource pool, then elastic resource scheduling can be performed, otherwise, resource elastic scheduling cannot be performed, and the current business queue needs to wait for an idle executor to execute the task;
[0098] 4) Perform resource scheduling on the current business queue and the executors in the dynamic resource pool to ensure that the current business queue has sufficient resources. Then use the idle executors to process the corresponding chip test tasks. The task processing results can be viewed in the corresponding directory later.
[0099] Figure 4 A schematic diagram of a resource scheduling device provided in an embodiment of the present invention, which is applied to a workload scheduler, includes:
[0100] The resource judgment module 11 is used to determine the first execution machine state of the current service queue and the second execution machine state in the dynamic resource pool, and determine whether the preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state;
[0101] An index generating module 12 is used for obtaining task information of chip test tasks in the current service queue if the preset resource mismatch condition is met, and performing weighted summation on the task information to generate a comprehensive load index of the current service queue;
[0102] An execution machine evaluation module 13 is used to determine the size relationship between the comprehensive load index and the current index threshold of the current service queue, and generate an execution machine usage evaluation result based on the size relationship;
[0103] The resource scheduling module 14 is used to determine a target resource scheduling strategy according to the evaluation result of the execution machine usage and the state of the second execution machine, and adjust the number of execution machines in the current service queue according to the target resource scheduling strategy.
[0104] It can be seen that the present invention is applied to a workload scheduler to determine the first execution machine state of the current business queue and the second execution machine state in the dynamic resource pool, and determine whether a preset resource mismatch condition is currently met based on the first execution machine state and the second execution machine state; if the preset resource mismatch condition is met, the task information of the chip test task in the current business queue is obtained, and the task information is weighted and summed to generate a comprehensive load index of the current business queue; determine the size relationship between the comprehensive load index and the current index threshold of the current business queue, and generate an execution machine usage evaluation result based on the size relationship; determine the target resource scheduling strategy based on the execution machine usage evaluation result and the second execution machine state, and adjust the number of execution machines in the current business queue according to the target resource scheduling strategy.
[0105] Beneficial effect: When a resource mismatch occurs between a business queue and a resource pool, the workload scheduler (i.e., OpenLava) of the present invention generates an executor usage evaluation result of the current business queue based on the task information of the chip test task in the current business queue, and adjusts the number of executors in the current business queue according to the target resource scheduling strategy determined according to the executor usage evaluation result and the state of the second executor. That is to say, the present invention monitors the business queue in real time, and when a resource mismatch occurs, intelligently generates a target resource scheduling strategy, and automatically schedules the resources of the business queue to ensure reasonable resource scheduling between the business queue and the resource pool, thereby ensuring task processing efficiency, and eliminating the need for manual resource configuration, thereby reducing management costs.
[0106] Figure 5 A structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 5 As shown, the electronic device includes: a memory 20 for storing a computer program;
[0107] The processor 21 is used to implement the steps of the resource scheduling method in the above embodiment when executing a computer program.
[0108] The electronic device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.
[0109] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.
[0110] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, the relevant steps of the resource scheduling method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include but is not limited to data transmitted by an external device received by the electronic device, and may also include data collected by its own input and output interface 23, etc.
[0111] In some embodiments, the electronic device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power source 25 , and a communication bus 26 .
[0112] Those skilled in the art will understand that Figure 5 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure.
[0113] It is understandable that if the resource scheduling method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the current technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk or optical disk and other media that can store program codes.
[0114] Based on this, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the resource scheduling method described above are implemented.
[0115] The above is a detailed introduction to a resource scheduling method, device, equipment and medium provided in an embodiment of the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0116] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0117] The above is a detailed introduction to a resource scheduling method, device, equipment and medium provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.
Claims
1. A resource scheduling method, characterized in that: Applicable to workload schedulers, including: Determine a first execution machine state of a current service queue and a second execution machine state in a dynamic resource pool, and determine whether a preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state; If the preset resource mismatch condition is met, task information of the chip test task in the current service queue is obtained, and weighted summation of the task information is performed to generate a comprehensive load index of the current service queue; Determine a magnitude relationship between the comprehensive load index and a current index threshold of the current service queue, and generate an execution machine usage evaluation result based on the magnitude relationship; A target resource scheduling strategy is determined according to the evaluation result of the use of the execution machine and the state of the second execution machine, and the number of execution machines in the current service queue is adjusted according to the target resource scheduling strategy.
2. The resource scheduling method according to claim 1, characterized in that: The determining whether a preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state includes: If the first execution machine state of the current business queue indicates that the number of execution machines in the current business queue that are in an idle state is less than a first preset threshold and the second execution machine state in the dynamic resource pool indicates that the number of execution machines in the dynamic resource pool that are in an idle state is greater than a second preset threshold, it is determined that the first preset resource mismatch condition is currently met; If the first execution machine state of the current business queue indicates that the number of execution machines in the current business queue that are in an idle state is greater than the third preset threshold and the second execution machine state in the dynamic resource pool indicates that the number of execution machines in the dynamic resource pool that are in an idle state is less than the fourth preset threshold, it is determined that the second preset resource mismatch condition is currently met; Correspondingly, the step of obtaining the task information of the chip test task in the current service queue includes: If the first preset resource mismatch condition is currently met, the target chip test task in the current business queue is suspended, and the task information of all chip test tasks in the current business queue is obtained; wherein the target chip test task is a task to which an execution machine is not assigned.
3. The resource scheduling method according to claim 2, characterized in that: The obtaining task information of the chip test task in the current service queue includes: Collecting the number of tasks of the current service queue and the maximum queue capacity of the current service queue, and determining the task queue length index of the chip test task based on the ratio between the number of tasks and the maximum queue capacity; The average task execution time of the current service queue is collected, and the task execution time index of the chip test task is determined based on the ratio between the average task execution time and a preset benchmark execution time.
4. The resource scheduling method according to claim 3, characterized in that: The weighted summing of the task information to generate a comprehensive load index of the current service queue includes: Determine a first weight coefficient of the task queue length index and a second weight coefficient of the task execution time index; Performing a weighted summation of the task queue length index and the task execution time index based on the first weight coefficient and the second weight coefficient to generate a comprehensive load index of the current service queue; Accordingly, determining the magnitude relationship between the comprehensive load index and the current index threshold of the current service queue, and generating an execution machine usage evaluation result based on the magnitude relationship, includes: Determine a current index threshold of the current service queue according to historical task execution data of the current service queue and system performance requirements; If the comprehensive load index is less than the current index threshold, generating an executor usage evaluation result indicating that the current service queue needs to increase the number of executors; If the comprehensive load index is not less than the current index threshold, an executor usage evaluation result is generated, indicating that the current service queue needs to reduce the number of executors.
5. The resource scheduling method according to claim 4, characterized in that: The determining a target resource scheduling strategy according to the execution machine usage evaluation result and the second execution machine state, and adjusting the number of execution machines of the current service queue according to the target resource scheduling strategy, includes: If the evaluation result of the execution machine usage indicates that the current business queue needs to increase the number of execution machines and the second execution machine state indicates that the number of execution machines in the dynamic resource pool that are idle is greater than the second preset threshold, then the target resource scheduling strategy is determined to be the target resource capacity expansion strategy, and the execution machines in the dynamic resource pool are included in the current business queue according to the target resource capacity expansion strategy to increase the number of execution machines in the current business queue; If the evaluation result of the execution machine usage indicates that the current business queue needs to reduce the number of execution machines and the second execution machine state indicates that the number of execution machines in the dynamic resource pool is less than the fourth preset threshold, then the target resource scheduling strategy is determined to be the target resource capacity reduction strategy, and the idle execution machines in the current business queue are included in the dynamic resource pool according to the target resource capacity reduction strategy to reduce the number of execution machines in the current business queue.
6. The resource scheduling method according to claim 1, characterized in that: Also includes: Get the current operation instruction based on the preset interface; If the current operation instruction is a viewing instruction, the target display information corresponding to the current operation instruction is filtered out from the current status and configuration information of each execution machine in the current business queue, the current status and configuration information of each execution machine in the dynamic resource pool, and the target resource scheduling strategy, and the target display information is displayed on a preset display interface; If the current operation instruction is a resource adjustment instruction, the execution machines in the current service queue and the execution machines in the dynamic resource pool are adjusted according to the current operation instruction.
7. The resource scheduling method according to claim 1, characterized in that: The determining of the target resource scheduling strategy according to the execution machine usage evaluation result and the second execution machine state includes: Determining the priority of each chip test task in the current service queue; A target resource scheduling strategy is determined according to the execution machine usage evaluation result, the second execution machine state, and the priority of each of the chip test tasks.
8. A resource scheduling device, characterized in that: Applicable to workload schedulers, including: A resource judgment module, used to determine the first execution machine state of the current service queue and the second execution machine state in the dynamic resource pool, and determine whether a preset resource mismatch condition is currently met according to the first execution machine state and the second execution machine state; An index generating module, configured to obtain task information of chip test tasks in the current service queue if the preset resource mismatch condition is met, and weighted sum the task information to generate a comprehensive load index of the current service queue; An execution machine evaluation module, used to determine the size relationship between the comprehensive load index and the current index threshold of the current service queue, and generate an execution machine usage evaluation result based on the size relationship; A resource scheduling module is used to determine a target resource scheduling strategy according to the evaluation result of the use of the execution machine and the state of the second execution machine, and adjust the number of execution machines in the current business queue according to the target resource scheduling strategy.
9. A resource scheduling device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the resource scheduling method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the resource scheduling method according to any one of claims 1 to 7 are implemented.
