Cluster scheduling method and system

Abstract

The invention discloses a cluster scheduling method and system.The cluster scheduling method comprises the steps that after a client takes over distributed cluster resource nodes distributed with standard unit feature extraction tasks, the client starts a monitoring thread to monitor the resource states of the nodes; the client judges whether the node is in an idle state according to the resource state; if yes, the client side sends the idle state to the server side, so that the server side judges whether to release the node or not according to the task priority and sends a corresponding instruction to the client side; and the client executes corresponding operation according to the instruction. According to the scheme, the execution efficiency of the standard unit feature extraction task can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of integrated circuit design, and specifically to a cluster scheduling method and system. Background Art

[0002] In integrated circuit design, the development of standard cell libraries is a key link in chip design. Standard cell libraries are the basic building blocks in chip design, and contain a variety of basic circuit units such as logic gates, flip-flops, and registers. In the chip design process, standard cell feature extraction is the process used to generate important data such as the electrical characteristics and timing characteristics of these units. Due to the wide application of these standard cells in circuits, the task of standard cell feature extraction has an important impact on the efficiency of the entire chip design.

[0003] However, in the development of standard cell libraries, the process of standard cell feature extraction often occupies more than two-thirds of the entire development cycle. This process not only has a large number of tasks, but also each task usually has a long task running time, which brings a significant time bottleneck to the development of standard cell libraries. In current integrated circuit design, as the functions of chips become increasingly complex, the task of standard cell feature extraction has become more complex, and the amount of calculation and processing requirements involved are constantly increasing. In order to meet the needs of efficient generation of standard cell library data, it is necessary to further improve the execution efficiency of standard cell feature extraction tasks. Summary of the invention

[0004] The embodiments of the present application provide a cluster scheduling method and system, which can improve the execution efficiency of standard cell feature extraction tasks.

[0005] In a first aspect, an embodiment of the present application provides a cluster scheduling method, including:

[0006] After the client takes over the distributed cluster resource node to which the standard unit feature extraction task has been assigned, the client starts a monitoring thread to monitor the resource status of the node;

[0007] The client determines whether the node is in an idle state according to the resource state;

[0008] If yes, the client sends the idle state to the server, so that the server determines whether to release the node according to the task priority and sends a corresponding instruction to the client;

[0009] The client performs corresponding operations according to the instructions.

[0010] In the cluster scheduling method provided in the embodiment of the present application, the client determines whether the node is in an idle state according to the resource state, including:

[0011] The client determines the CPU utilization and task allocation status of the node within a first preset time according to the resource;

[0012] When the node is not assigned a standard cell feature extraction task within the first preset time and the CPU utilization is lower than a preset threshold, it is determined that the node is in an idle state.

[0013] In the cluster scheduling method provided in the embodiment of the present application, the resource status includes memory consumption, task running time and task quantity, and the cluster scheduling method further includes:

[0014] The client performs threshold detection on the memory consumption, task running time and task quantity of the node, and determines whether to release the node based on the detection.

[0015] In the cluster scheduling method provided in the embodiment of the present application, the server determines whether to release the node according to the task priority and sends a corresponding instruction to the client, including:

[0016] When the server receives the idle state, the server determines whether there is a high-priority standard cell feature extraction task to be assigned in the task execution list;

[0017] If yes, the server determines to retain the node and sends a retention instruction to the client;

[0018] If not, the server determines to release the node and sends a release instruction to the client.

[0019] The cluster scheduling method provided in the embodiment of the present application also includes:

[0020] The client starts a heartbeat thread and sends the resource status of the node to the server;

[0021] The server determines whether the node is abnormal according to the resource status;

[0022] When the node is abnormal, the server reallocates the standard unit feature extraction task of the node.

[0023] In the cluster scheduling method provided in the embodiment of the present application, after the server reallocates the standard unit feature extraction task of the node when the node is abnormal, the method further includes:

[0024] The server determines whether the standard cell feature extraction task is a high priority task;

[0025] When the standard cell feature extraction task is a high priority task, it is determined to occupy a new node to execute the abnormal standard cell feature.

[0026] In the cluster scheduling method provided in the embodiment of the present application, when the standard cell feature extraction task is not a high priority task, the server obtains the number of tasks to be assigned to the standard cell feature extraction task in the task execution list, and determines whether to occupy a new node based on the number of tasks.

[0027] In the cluster scheduling method provided in the embodiment of the present application, determining whether to occupy a new node according to the number of tasks includes:

[0028] Determine whether the currently occupied node has execution capability according to the number of tasks and the resource status;

[0029] If not, it is determined to occupy a new node to execute the abnormal standard cell feature.

[0030] The cluster scheduling method provided in the embodiment of the present application also includes:

[0031] When the server does not receive the resource status within the second preset time, it is determined that the node is abnormal.

[0032] In a second aspect, an embodiment of the present application provides a cluster scheduling system, including a client and a server, wherein the client communicates with the server via grpc to implement any of the cluster scheduling methods described above.

[0033] In summary, the cluster scheduling method provided by the embodiment of the present application includes that after the client takes over the distributed cluster resource node to which the standard cell feature extraction task has been assigned, the client starts a monitoring thread to monitor the resource status of the node; the client determines whether the node is in an idle state based on the resource status; if so, the client sends the idle state to the server, so that the server determines whether to release the node based on the task priority, and sends a corresponding instruction to the client; the client performs a corresponding operation based on the instruction. In this solution, after receiving the instruction from the server, the client decides whether to release the resources or continue to occupy the current node according to the specific situation, ensuring that the task is not delayed due to the existence of idle nodes during the task allocation process, so that the cluster system can respond quickly when facing a surge in task volume or resource shortages, reduce task waiting time, and thus improve the efficiency of task execution. That is, this solution can improve the execution efficiency of standard cell feature extraction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 It is a flow chart of the cluster scheduling method provided in the embodiment of the present application.

[0036] Figure 2 It is another flow chart of the cluster scheduling method provided in an embodiment of the present application.

[0037] Figure 3 It is a structural diagram of the cluster scheduling system provided in an embodiment of the present application.

[0038] Figure 4 It is a structural diagram of the client or server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0040] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0041] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0043] In the description of the present application, it should be noted that the terms "upper", "lower", "left", "right", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, terms such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0044] In the development of standard cell libraries, the process of standard cell feature extraction often occupies more than two-thirds of the entire development cycle. This process not only involves a large number of tasks, but also the task running time of each task is usually long, which brings a significant time bottleneck to the development of standard cell libraries. In current integrated circuit design, as the functions of chips become increasingly complex, the task of standard cell feature extraction has become more complex, and the amount of calculation and processing requirements involved are constantly increasing. In order to meet the needs of efficient generation of standard cell library data, it is necessary to further improve the execution efficiency of standard cell feature extraction tasks.

[0045] Based on this, the embodiment of the present application provides a cluster scheduling method and system. The technical solutions shown in the present application will be described in detail below through specific embodiments. It should be noted that the description order of the following embodiments is not intended to limit the priority order of the embodiments.

[0046] See also Figure 1 , Figure 1 : is a flow chart of the cluster scheduling method provided in the embodiment of the present application. The specific flow of the cluster scheduling method can be as follows:

[0047] 101. After the client takes over the distributed cluster resource node to which the standard unit feature extraction task has been assigned, the client starts a monitoring thread to monitor the resource status of the node.

[0048] Among them, the distributed cluster resource node refers to an independent computing unit that undertakes functions such as computing, storage, and task execution in a distributed computing system. These nodes usually coordinate work in a cluster system and jointly undertake a large number of computing tasks. Each node will run a specific task or multiple tasks. In an embodiment of the present application, a distributed cluster resource node refers to a node that has been assigned a standard cell feature extraction task.

[0049] After the cluster resources are scheduled, the currently started distributed cluster resource nodes can be handed over to the client for management. After the client takes over the distributed cluster resource node that has been assigned the standard unit feature extraction task, the client can start the monitoring thread to monitor the resource status of the node. It should be noted that the monitoring thread runs independently of other tasks to ensure that the resource status of the node can be continuously monitored.

[0050] The resource status may include parameters such as CPU utilization, memory consumption, task running time, and task quantity.

[0051] In some embodiments, the client may monitor the resource status of the node in a controllable clock monitoring manner, that is, the client may obtain the resource status of the node at a first preset time interval.

[0052] 102. The client determines whether the node is in an idle state according to the resource status.

[0053] The idle state refers to the state in which no tasks are assigned to a node for a certain period of time and its CPU utilization is not effectively utilized. When a node is in the idle state, its computing resources are not effectively utilized, which may lead to resource waste.

[0054] In some embodiments, the client can determine the CPU utilization and task allocation status of the node within a first preset time based on resources; when the node is not assigned a standard unit feature extraction task within the first preset time and the CPU utilization is lower than a preset threshold, it is determined that the node is in an idle state.

[0055] Among them, the task allocation status can be determined based on the task running time and the task quantity.

[0056] 103. If so, the client sends the idle state to the server, so that the server determines whether to release the node according to the task priority and sends a corresponding instruction to the client.

[0057] Specifically, when the server receives the idle status, the server determines whether there is a high-priority standard unit feature extraction task to be assigned in the task execution list; if so, the server determines to retain the node and sends a retention instruction to the client; if not, the server determines to release the node and sends a release instruction to the client.

[0058] In an embodiment of the present application, the server may prioritize each standard cell feature extraction task in the task execution list. For example, each standard cell feature extraction task in the task execution list may be labeled as a "high priority task", "medium priority task" and "low priority task".

[0059] It is understandable that if there is a high-priority task to be assigned in the task execution list, the server may consider that the node may still be used to process the high-priority task to be assigned. Therefore, the server may determine to retain the node. If there is no high-priority task to be assigned in the task execution list, it means that there is no urgent task to be executed, so the server may determine to release the node.

[0060] 104. The client performs corresponding operations according to the instructions.

[0061] Specifically, when the client receives a reserve instruction, the node is reserved, and when the client receives a release instruction, the node is released.

[0062] By releasing idle nodes, the computing resources of the cluster can be used more effectively. It is inefficient for nodes to occupy resources when there are no tasks, so releasing these nodes can free up resources for other tasks, thereby improving the overall resource utilization of the cluster. In addition, when the nodes are idle, no tasks are executed and the resources are idle. If these nodes are not released in time, it will cause a waste of resources. By releasing idle nodes in time, this waste can be avoided and computing resources can be allocated and used more efficiently.

[0063] Understandably, if the memory consumption of a node exceeds a threshold and there are no valid tasks currently running, the node can be considered to be in a high-load state and needs to be released. If the running time of tasks on a node is too long and the number of tasks is not properly managed, it may cause the efficiency of the node to decrease and resources need to be released. If the number of tasks on a node is too large and the tasks are not completed on time, it may mean that the node is overloaded and cannot run efficiently.

[0064] Therefore, in some embodiments, the client may also perform threshold detection on the memory consumption, task running time and number of tasks of the node, and determine whether to release the node based on the detection.

[0065] The client determines whether a node is overloaded by comprehensively judging memory consumption, task running time, and number of tasks. If any of these three indicators exceeds the preset threshold and the node has no valid tasks to execute, the client can release the node. By promptly releasing node resources with excessive memory consumption, too many tasks, or too long running time, system overload can be avoided and the overall cluster performance can be improved.

[0066] In summary, the cluster scheduling method provided by the embodiment of the present application includes that after the client takes over the distributed cluster resource node to which the standard cell feature extraction task has been assigned, the client starts a monitoring thread to monitor the resource status of the node; the client determines whether the node is in an idle state based on the resource status; if so, the client sends the idle state to the server, so that the server determines whether to release the node based on the task priority, and sends the corresponding instruction to the client; the client performs the corresponding operation according to the instruction. In this scheme, after receiving the instruction from the server, the client decides whether to release the resources or continue to occupy the current node according to the specific situation, ensuring that the task is not delayed due to the existence of idle nodes during the task allocation process, so that the cluster system can respond quickly when facing a surge in task volume or resource shortages, reduce task waiting time, and thus improve the efficiency of task execution. That is, this scheme can improve the execution efficiency of standard cell feature extraction tasks.

[0067] In order to better implement the cluster scheduling method provided in the embodiment of the present application, the embodiment of the present application also provides a cluster scheduling method, wherein the meanings of the terms are the same as those in the above cluster scheduling method, and the specific implementation details can refer to the description in the method embodiment.

[0068] like Figure 2 As shown, the specific process of the cluster scheduling method can be as follows:

[0069] 201. The client starts the heartbeat thread and sends the resource status of the node to the server.

[0070] The heartbeat thread is usually started after the client takes over the distributed cluster resource node to which the standard cell feature extraction task has been assigned, to ensure that the client can continuously track the resource status of the node. That is, the execution order of step 201 is after step 101 of the above embodiment.

[0071] Through the continuous heartbeat mechanism, the server can understand the resource status of each node in real time, avoid resource waste or overload, and enable task scheduling to be dynamically adjusted according to the actual load of the node, thereby improving the operation efficiency of the cluster.

[0072] 202. The server determines whether the node is abnormal based on the resource status.

[0073] The resource status may include parameters such as CPU utilization, memory consumption, task running time, and task quantity.

[0074] Specifically, if the CPU usage rate is at an extreme value (such as 100% or 0%) for a long time, and the tasks on the node do not show any progress or signs of normal operation, the server will determine that the node may be abnormal. If the memory usage of the node exceeds a predetermined threshold (such as 80% or 90%), the server may consider that the node may have a memory overflow or memory leak problem, and mark it as abnormal. If the tasks on the node continue to be incomplete and the node's resource consumption is also high, it may mean that the node has entered a deadlock or the task execution is abnormal, and the server should regard it as an abnormal node.

[0075] In one embodiment, when the server does not receive the resource status within the second preset time, it can be determined that the node is abnormal.

[0076] 203. When a node is abnormal, the server reallocates the standard unit feature extraction task of the node.

[0077] After the server reallocates the standard cell feature extraction task of the node, the server can first determine whether the standard cell feature extraction task is a high priority task. When the standard cell feature extraction task is a high priority task, it is determined to occupy a new node to execute the abnormal standard cell feature. When the standard cell feature extraction task is not a high priority task, the server obtains the number of tasks to be assigned to the standard cell feature extraction task in the task execution list, and determines whether to occupy a new node based on the number of tasks.

[0078] Specifically, it can be determined whether the currently occupied node has the execution capability according to the number of tasks and the resource status; if not, it is determined to occupy a new node to execute the abnormal standard unit feature.

[0079] For example, if there are a large number of tasks to be assigned and the currently occupied node cannot meet the task requirements, the server can determine to occupy a new node. If the resource consumption of the currently occupied node is close to saturation or has exceeded the set threshold and cannot be expanded, the server can determine to occupy a new node.

[0080] It should be noted that the new node refers to the unoccupied node in the current cluster.

[0081] In summary, in the cluster scheduling method provided in this embodiment, through the heartbeat mechanism, the server can promptly detect the abnormal state of the node, and by reallocating tasks, ensure that the task continues to be executed, and the task will not fail due to the failure of a certain node. Even if some nodes are abnormal, the entire cluster system can still maintain stable operation, thereby improving the robustness of the system. When the server detects a node abnormality, it will automatically reallocate the tasks on the abnormal node to other healthy nodes, which ensures that the task can continue to be executed and completed on time, avoiding task interruptions or delays caused by node failures. That is, this solution can improve cluster resource utilization, enhance system fault tolerance, improve task scheduling and execution efficiency, and reduce the complexity of cluster management through mechanisms such as real-time monitoring of node resource status, intelligent task allocation, timely detection of abnormal nodes, and task migration.

[0082] The present application also provides a cluster scheduling system. Figure 3 As shown, the cluster scheduling system may include a client and a server. The client and the server communicate via grpc to implement the cluster scheduling method in the above embodiment.

[0083] grpc (Google Remote Procedure Call) is a high-performance, open source and universal remote procedure call framework developed by Google. It enables applications on different platforms and in different programming languages ​​to communicate efficiently and reliably over the network. gRPC is based on the HTTP / 2 protocol and can support multiple communication modes, such as one-way calls, two-way streams, client streams, and server streams. gRPC is often used in microservice architectures, especially for scenarios that require fast, efficient, and low-latency communication.

[0084] like Figure 4 As shown, it shows a schematic diagram of the structure of the client or server involved in the embodiment of the present application, specifically:

[0085] The client or server may include one or more processors 301 of processing cores and one or more computer-readable storage media memories 302 and other components. Those skilled in the art will appreciate that Figure 4 The client or server structure shown in the figure does not constitute a limitation on the client or server, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0086] in:

[0087] The processor 301 is the control center of the client or server, and uses various interfaces and lines to connect the various parts of the entire client or server. By running or executing the software program and / or the present application stored in the memory 302, and calling the data stored in the memory 302, it executes various functions of the client or server and processes data, thereby monitoring the client or server as a whole. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operation storage medium, user interface and application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0088] The memory 302 can be used to store software programs and the present application. The processor 301 executes various functional applications and data processing by running the software programs and the present application stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store operating storage media, at least one application required for a function, etc.; the data storage area may store data created according to the use of the client or server, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0089] Although not shown, the client or server may also include a display unit, an input unit, a power supply, etc., which are not described in detail herein. Specifically in this embodiment, the processor 301 in the client or server loads the executable files corresponding to the processes of one or more applications into the memory 302 according to the following instructions, and the processor 301 runs the applications stored in the memory 302, thereby realizing various functions.

[0090] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0091] To this end, an embodiment of the present application provides a storage medium, in which multiple instructions are stored. The instructions can be loaded by a processor to execute the steps in any cluster scheduling method provided in the embodiment of the present application.

[0092] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0093] Since the instructions stored in the storage medium can execute the steps in any method provided in the embodiments of the present application, the beneficial effects that can be achieved by any method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0094] The cluster scheduling method and system provided by the present application are respectively introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the core idea of ​​the present application; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A cluster scheduling method, characterized in that: include: After the client takes over the distributed cluster resource node to which the standard unit feature extraction task has been assigned, the client starts a monitoring thread to monitor the resource status of the node; The client determines whether the node is in an idle state according to the resource state; If yes, the client sends the idle state to the server, so that the server determines whether to release the node according to the task priority and sends a corresponding instruction to the client; The client performs corresponding operations according to the instructions.

2. The cluster scheduling method according to claim 1, characterized in that: The client determines whether the node is in an idle state according to the resource state, including: The client determines the CPU utilization and task allocation status of the node within a first preset time according to the resource; When the node is not assigned a standard cell feature extraction task within the first preset time and the CPU utilization is lower than a preset threshold, it is determined that the node is in an idle state.

3. The cluster scheduling method according to claim 1, characterized in that: The resource status includes memory consumption, task running time and task quantity. The cluster scheduling method further includes: The client performs threshold detection on the memory consumption, task running time and task quantity of the node, and determines whether to release the node based on the detection.

4. The cluster scheduling method according to claim 1, characterized in that: The server determines whether to release the node according to the task priority and sends a corresponding instruction to the client, including: When the server receives the idle state, the server determines whether there is a high-priority standard cell feature extraction task to be assigned in the task execution list; If yes, the server determines to retain the node and sends a retention instruction to the client; If not, the server determines to release the node and sends a release instruction to the client.

5. The cluster scheduling method according to claim 1, characterized in that: Also includes: The client starts a heartbeat thread and sends the resource status of the node to the server; The server determines whether the node is abnormal according to the resource status; When the node is abnormal, the server reallocates the standard unit feature extraction task of the node.

6. The cluster scheduling method according to claim 5, characterized in that: After the server reallocates the standard unit feature extraction task of the node when the node is abnormal, the method further includes: The server determines whether the standard cell feature extraction task is a high priority task; When the standard cell feature extraction task is a high priority task, it is determined to occupy a new node to execute the abnormal standard cell feature.

7. The cluster scheduling method according to claim 6, characterized in that: When the standard cell feature extraction task is not a high priority task, the server obtains the number of tasks to be assigned the standard cell feature extraction task in the task execution list, and determines whether to occupy a new node according to the number of tasks.

8. The cluster scheduling method according to claim 7, characterized in that: The determining whether to occupy a new node according to the number of tasks includes: Determine whether the currently occupied node has execution capability according to the number of tasks and the resource status; If not, it is determined to occupy a new node to execute the abnormal standard cell feature.

9. The cluster scheduling method according to claim 5, characterized in that: Also includes: When the server does not receive the resource status within the second preset time, it is determined that the node is abnormal.

10. A cluster scheduling system, characterized in that: It includes a client and a server, and the client communicates with the server through grpc to implement the cluster scheduling method according to any one of claims 1 to 9.

Images