Task scheduling method, apparatus and system, storage medium, and computer program product

Abstract

The present application relates to the technical field of computers, and discloses a task scheduling method, apparatus and system, a storage medium, and a computer program product. The method comprises: after a resource allocation requirement of a task to be executed is acquired, if a historical resource allocation requirement that is the same as the resource allocation requirement of said task is recorded in historical resource allocation information, directly determining an execution node that is recorded in the historical resource allocation information and matches the historical resource allocation requirement as a target execution node. In this way, there is no need to re-determine from among multiple execution nodes an execution node of which a node label matches the resource allocation requirement of said task, and a large amount of string matching calculation is not required, such that the screening efficiency of execution nodes is improved, thereby improving the scheduling efficiency and the execution efficiency of tasks.

Description

Task scheduling method, device, system, storage medium and computer program product

[0001] This application claims priority to the Chinese patent application with application number 202311191240.X filed on September 13, 2023, and with invention name “Task scheduling method, device, system, storage medium and computer program product”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a task scheduling method, device, system, storage medium, and computer program product. Background Art

[0003] Currently, users can encapsulate business logic into tasks through the client and submit relevant task information to the management node in the task scheduling system. The task scheduling system also includes multiple execution nodes. The management node is responsible for scheduling each task submitted by the user to at least one of the multiple execution nodes according to the scheduling policy, so that the at least one execution node can execute the task. Reasonable and efficient scheduling methods can improve the resource utilization of the task scheduling system and enhance the efficiency of task execution.

[0004] With the development of scheduling technology, node label-based scheduling technology has emerged. In related technologies for task scheduling based on node labels, each execution node has a node label that represents one or more information about the corresponding execution node, such as the central processing unit (CPU) architecture, operating system type, whether it includes heterogeneous processors, and whether it includes a graphics processing unit (GPU). When encapsulating a task, users can specify a requirement label. In this way, the relevant information of the task submitted by the user includes the requirement label. After receiving a task with a requirement label, the management node matches the requirement label with the node labels of each of the multiple execution nodes through string comparison, thereby filtering out the execution nodes that match the task and then scheduling the task to the filtered execution nodes. For example, if the requirement label of a task submitted by a user includes 'x86', indicating that the user wishes to execute the task on an execution node with an x86 CPU architecture, the management node first filters out all execution nodes with an x86 CPU architecture in the task scheduling system through string comparison, and then filters out the execution node from all execution nodes with an x86 CPU architecture to execute the task.

[0005] However, the task scheduling system must handle a large number of tasks and execution nodes. The above method requires comparing each task's requirement label with the node label of each execution node, resulting in low screening efficiency and impacting scheduling efficiency. Furthermore, the computational complexity of screening execution nodes through string comparison is high, which also affects scheduling efficiency and, consequently, task execution efficiency.

[0006] Summary of the Invention

[0007] This application provides a task scheduling method, device, storage medium, and computer program product that can improve scheduling efficiency and task execution efficiency. The technical solution is as follows:

[0008] In a first aspect, a task scheduling method is provided, which is applied to a management node in a task scheduling system, wherein the task scheduling system also includes multiple execution nodes. The method includes:

[0009] Obtain the resource allocation requirements of the task to be executed; confirm whether there is a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed in the historical resource allocation information recorded by the management node, and the historical resource allocation information is used to record the information of the execution node that matches the resource allocation requirement of the executed task by the management node; when it is determined that there is a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed in the historical resource allocation information, determine the execution node that matches the historical resource allocation requirement recorded in the historical resource allocation information as the target execution node; schedule the task to be executed to the target execution node for execution.

[0010] That is, if the historical resource allocation information records historical resource allocation requirements that are the same as the resource allocation requirements of the task to be executed, the execution node recorded in the historical resource allocation information that matches the historical resource allocation requirements is directly determined as the target execution node. In this way, there is no need to re-determine the execution node whose node label matches the resource allocation requirements of the task to be executed from multiple execution nodes, and there is no need for a large amount of string matching calculations, which improves the screening efficiency of the execution nodes, thereby improving the scheduling efficiency and the execution efficiency of the task.

[0011] Optionally, the resource allocation requirement includes multiple sub-requirements, which are expressed as logical expressions. Compared to expressing resource allocation requirements as a set of sub-tags, expressing resource allocation requirements as a logical expression can meet more user needs and improve the usability and functional completeness of the task scheduling system.

[0012] Optionally, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold. And / or, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a second threshold. This improves the hit rate of obtaining a target execution node from the historical resource allocation information.

[0013] Optionally, the resource allocation requirements of the task to be executed are represented by a first requirement tag, which is expressed as an infix expression. Infix expressions are logical expressions similar to AND, OR, and NOT. Compared to representing requirement tags as subtag sets, using AND, OR, and NOT logical expressions can meet more user needs and improve the usability and functional completeness of the task scheduling system.

[0014] Since a requirement tag with the same semantics as a requirement tag may have multiple representations, in order to unify the representations for label matching, certain rules can be preset so that these multiple representations can be converted into a unique representation according to the preset rules.

[0015] Optionally, the method further includes: converting the first requirement tag into a second requirement tag according to a preset rule, where the second requirement tag and the first requirement tag have the same semantics but a different representation; and if the second requirement tag is recorded in the historical resource allocation information, determining that the historical resource allocation information records a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed. Performing unified tag conversion according to the preset rules can significantly reduce computational complexity and improve cache hit rates.

[0016] If the first requirement tag is expressed as an infix expression, the second requirement tag is expressed as a postfix expression. Postfix expressions are also called reverse Polish notation, which is easier for computers to understand and calculate.

[0017] Optionally, the method further includes: if the second requirement tag is not recorded in the historical resource allocation information, determining an execution node whose node tag matches the second requirement tag from the multiple execution nodes; and determining the execution node whose node tag matches the second requirement tag from the multiple execution nodes as the target execution node.

[0018] The second requirement tag includes at least one requirement subtag; determining the execution node whose node tag matches the second requirement tag from the multiple execution nodes includes: obtaining a binary string corresponding to at least one requirement subtag from a node bitmap, the node bitmap includes binary strings corresponding to multiple reference tags, each binary string includes a multi-bit binary number corresponding to the multiple execution nodes, each binary number indicates whether the corresponding execution node has a corresponding reference tag; replacing at least one requirement subtag in the target suffix expression with a corresponding binary string to obtain an expression to be operated, the target suffix expression refers to a suffix expression used to represent the second requirement tag; determining the value of the expression to be operated by bit operation to obtain a target binary string; determining the execution node with the corresponding reference tag in the target binary string as the execution node whose node tag matches the second requirement tag among the multiple execution nodes. The use of bit operations and reverse Polish notation can convert label scheduling into efficient bit operations, thereby improving performance.

[0019] Optionally, a correspondence between the second requirement tag and the target binary string is added to the historical resource allocation information. By recording the binary string in the historical resource allocation information to represent the corresponding execution node, the amount of stored data can be reduced.

[0020] Optionally, historical resource allocation information is recorded in a cache of the management node, which can improve scheduling efficiency.

[0021] Optionally, the method also includes: obtaining the node label of each execution node among the multiple execution nodes; generating a node bitmap based on the node label of each execution node among the multiple execution nodes and multiple reference labels, wherein the node bitmap represents whether each execution node among the multiple execution nodes has each reference label among the multiple reference labels.

[0022] Wherein, obtaining the node label of each execution node among the multiple execution nodes includes: receiving first configuration information, the first configuration information includes at least two node sub-labels configured for the first execution node among the multiple execution nodes, the first execution node is any execution node whose node label includes the multiple node sub-labels; and / or, receiving second configuration information, the second configuration information includes node labels configured for at least two execution nodes among the multiple execution nodes. That is, this solution supports configuring multiple node sub-labels for one execution node at a time, and also supports configuring node labels for a batch of execution nodes at a time. In this way, the flexibility of administrators in configuring custom labels can be improved, and the usability and functional completeness of the task scheduling system can be improved.

[0023] Optionally, scheduling the task to be executed to the target execution node for execution includes: determining at least one first target execution node from the target execution nodes, and scheduling the task to be executed to the at least one first target execution node for execution. That is, the target execution node serves as a candidate execution node, and the management node further screens out at least one execution node from the candidate execution nodes to ultimately execute the task. Among them, further screening strategies may include load balancing strategies or random selection strategies, etc. The number of the at least one execution node can be determined according to the task amount of the task to be executed or the node number indication information of the task to be executed, or it can also be determined by the management node according to the default settings. These methods for determining the number of execution nodes can be used alone or in combination.

[0024] In a second aspect, a task scheduling device is provided, which has the function of implementing the task scheduling method described in the first aspect. The task scheduling device includes one or more modules, which are used to implement the task scheduling method described in the first aspect.

[0025] In a third aspect, a task scheduling system is provided, comprising a management node and a plurality of execution nodes. The management node is configured to execute the steps of the task scheduling method provided in the first aspect. The plurality of execution nodes are configured to execute tasks assigned by the management node.

[0026] In a fourth aspect, a computer device is provided, comprising a processor and a memory, wherein the memory is configured to store a program for executing the task scheduling method provided in the first aspect, and to store data involved in implementing the task scheduling method provided in the first aspect. The processor is configured to execute the program stored in the memory. The computer device may further comprise a communication bus for establishing a connection between the processor and the memory.

[0027] In a fifth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, which, when executed on a computer, enables the computer to execute the task scheduling method described in the first aspect.

[0028] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the task scheduling method described in the first aspect.

[0029] The technical effects obtained in the above-mentioned second to sixth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 is a system architecture diagram of a task scheduling method provided in an embodiment of the present application;

[0031] FIG2 is a system architecture diagram of another task scheduling method provided in an embodiment of the present application;

[0032] FIG3 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0033] FIG4 is a flowchart of a task scheduling method provided in an embodiment of the present application;

[0034] FIG5 is a schematic diagram of a reverse Polish notation represented by a tree structure provided in an embodiment of the present application;

[0035] FIG6 is a flow chart of a method for obtaining node labels and generating node bitmaps provided in an embodiment of the present application;

[0036] FIG7 is a flowchart of another task scheduling method provided in an embodiment of the present application;

[0037] FIG8 is a schematic structural diagram of a task scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0039] FIG1 is a diagram of the system architecture involved in a task scheduling method provided in an embodiment of the present application. The system may be referred to as a task scheduling system, or simply a scheduling system. Referring to FIG1 , the system includes a management node 101 and multiple execution nodes 102. Management node 101 and the multiple execution nodes 102 each have a wireless or wired communication connection.

[0040] The management node 101 is configured to obtain task information of a pending task, including resource allocation requirements, determine a target execution node 102 for the pending task using this solution, and schedule the pending task to be executed on the target execution node 102. Optionally, the management node 101 is configured to schedule the pending task to be executed on at least one of the target execution nodes 102.

[0041] Each execution node 102 in the plurality of execution nodes 102 is used to execute a task assigned by the management node 101 .

[0042] In an embodiment of the present application, the multiple execution nodes 102 can be multiple computing nodes, multiple network nodes, multiple computing network elements, multiple grids, or multiple computing processes. Network nodes can include network devices such as switches and routers. Computing network elements and grids can be understood as upper-level concepts of computing nodes. For example, a computing network element can include multiple computing nodes. A computing process can be a Linux process. That is, this solution can be applied to various scheduling scenarios, including various distributed cluster computing scenarios, such as high-performance computing (HPC) clusters, artificial intelligence (AI), big data, cloud services, microservice scheduling, network switching, and other scenarios.

[0043] Next, please refer to Figure 2 to introduce the modules included in the management node and the modules included in the execution node.

[0044] Figure 2 is a system architecture diagram of another task scheduling method provided in an embodiment of the present application. The system includes a management node and multiple execution nodes, and Figure 2 shows any one of the multiple execution nodes.

[0045] Referring to Figure 2 , the management node includes an application processing interface (API) interaction module, a resource management module, and a job scheduling module. In some embodiments of the present application, jobs may also be referred to as tasks. The API interaction module includes a resource management interface and a job submission interface. The job scheduling module may also be referred to as a scheduling engine, task scheduling module, or the like.

[0046] The resource management interface is for administrators to use. Administrators can use it to configure node labels for execution nodes. Specifically, the resource management interface is used to receive configuration information, including the node labels configured by the administrator for the execution nodes. Optionally, administrators can also use it to view, delete, and modify the node labels of execution nodes. Specifically, the resource management interface is used to receive node label query requests and / or node label update requests, and to return corresponding information.

[0047] The job submission interface is for users to use. Users can submit tasks through the job submission interface. The task information of the submitted task may include resource allocation requirements, which can be represented by a requirement tag. That is, the job submission interface is used to receive task information of the task, and the task information includes a requirement tag.

[0048] The resource management module is used to obtain the node label of each execution node in the multiple execution nodes, including obtaining the configuration information submitted by the administrator, and optionally, also includes the node information of the execution nodes automatically reported by the execution nodes or automatically collected by the resource management module, and obtains a part of the node sub-labels included in the node label from the node information. The resource management module is also used to generate a node bitmap based on the node label of each execution node in the multiple execution nodes and multiple reference labels, and synchronize the node bitmap to the job scheduling module. Among them, the node bitmap indicates whether each execution node in the multiple execution nodes has each reference label in the multiple reference labels. For specific implementation methods, please refer to the detailed description of the method embodiment below.

[0049] The job scheduling module is used to schedule tasks based on the above node bitmap using this solution, including determining the target execution node for the task to be executed, and scheduling the task to be executed to at least one execution node among the target execution nodes, that is, sending the task to the execution node.

[0050] The execution node includes a job management module and an operating system. The job management module is used to receive tasks issued by the management node, such as tasks issued by the job scheduling module, and run the received tasks in the operating system, that is, execute the corresponding tasks through the operating system.

[0051] Optionally, the execution node also includes a resource monitoring module. The resource monitoring module is used to collect node information of the execution node from the operating system. The node information includes but is not limited to the following information: node name, performance information, operating system information, processor information, etc., where the operating system information and processor information can be used as node subtags. The resource monitoring module is also used to proactively report the node information of the execution node to the management node, or send the node information of the execution node to the management node after receiving a request from the management node. As an example, the execution node sends node information to the management node when registering with the task scheduling system and / or when going online.

[0052] Optionally, the resource monitoring module collects node information from the operating system as raw information, and the resource monitoring system processes the raw information and reports it to the management node, for example, by unifying the format of the raw information and encapsulating it into a message format.

[0053] The embodiment of the present application also provides a scheduler, see Figure 2, the scheduler includes a management side and a computing side. Among them, the management side includes the resource management interface, job submission interface, resource management module and job scheduling module introduced above. The computing side includes the resource monitoring module and job management module introduced above. The scheduler can be regarded as the brain of the distributed computing cluster, which is mainly responsible for allocating corresponding system resources to the tasks submitted by users. As can be seen from Figure 2, the scheduler provides rich functions and interfaces, which are convenient for users to use and administrators to operate and maintain, and can improve the utilization of system resources, improve scheduling efficiency and task execution efficiency.

[0054] It should be understood that the system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0055] Please refer to Figure 3, which is a schematic diagram of the structure of a computer device according to an embodiment of the present application. Optionally, the computer device is the management node or execution node shown in Figure 1, and the computer device includes one or more processors 301, a communication bus 302, a memory 303, and one or more communication interfaces 304.

[0056] The processor 301 is a general-purpose central processing unit (CPU), a network processing processor (NP), a microprocessor, or one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. Optionally, the PLD is a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0057] Communication bus 302 is used to transmit information between the above components. Optionally, communication bus 302 is divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG3 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0058] Optionally, the memory 303 is a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compact disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 303 exists independently and is connected to the processor 301 via the communication bus 302, or the memory 303 is integrated with the processor 301.

[0059] The communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks. The communication interface 304 includes a wired communication interface and, optionally, a wireless communication interface. Examples of wired communication interfaces include Ethernet interfaces. Optionally, the Ethernet interface is an optical interface, an electrical interface, or a combination thereof. The wireless communication interface is a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0060] Optionally, in some embodiments, the computer device includes multiple processors, such as processor 301 and processor 305 shown in Figure 3. Each of these processors is a single-core processor or a multi-core processor. Optionally, the processor herein refers to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0061] In a specific implementation, as an embodiment, the computer device further includes an output device 306 and an input device 307. The output device 306 communicates with the processor 301 and can display information in a variety of ways. For example, the output device 306 is a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 307 communicates with the processor 301 and can receive user input in a variety of ways. For example, the input device 307 is a mouse, a keyboard, a touch screen device, or a sensor device.

[0062] In some embodiments, the memory 303 is used to store program code 310 for executing the solution of the present application, and the processor 301 is capable of executing the program code 310 stored in the memory 303. The program code includes one or more software modules, and the computer device can implement the task scheduling method provided in the embodiment of Figure 4 below through the processor 301 and the program code 310 in the memory 303.

[0063] FIG4 is a flowchart of a task scheduling method provided by an embodiment of the present application, which is applied to a management node in a task scheduling system, and the task scheduling system also includes multiple execution nodes.

[0064] Step 401: Obtain resource allocation requirements for tasks to be executed.

[0065] In an embodiment of the present application, the management node receives task information of a task to be executed, and the task information includes resource allocation requirements.

[0066] Among them, the resource allocation requirement includes one or more sub-requirements, and the one or more sub-requirements can be expressed in any form. For example, the one or more sub-requirements are expressed in the form of a sub-requirement set, and the sub-requirements in the sub-requirement set can be connected by commas, semicolons, spaces or other symbols. For another example, the one or more sub-requirements are expressed in the form of a logical expression, and the logical expression can be an infix expression, a postfix expression or an expression in other forms. The logical expression method can meet more user needs, and the user can set resource allocation requirements in a more flexible way.

[0067] In an embodiment of the present application, a resource allocation requirement may be represented by a first requirement tag. The first requirement tag includes at least one requirement subtag. The at least one requirement subtag may be represented in any form. One requirement subtag represents one subrequirement.

[0068] In one implementation, the at least one requirement subtag is in the form of a subtag set. For example, the first requirement tag is 'bigmem, npu, centos', that is, the first requirement tag includes three requirement subtags, and the subtag set formed by these three requirement subtags is the first requirement tag.

[0069] In another implementation, in order to improve the flexibility of users in tagging tasks, reduce the requirements for users to tag tasks, and meet more user needs, the first requirement label can be expressed in the form of an infix expression.

[0070] For example, the first requirement tag is 'bigmem', that is, the first requirement tag includes a requirement sub-tag, and the requirement sub-tag indicates that the user expects the task to be executed to be executed on an execution node with a large memory. It should be understood that when the first requirement tag includes a requirement sub-tag, the first requirement tag is also an infix expression. For another example, the first requirement tag is 'bigmem&npu&(centos|ubuntu)', that is, the first requirement tag includes four requirement sub-tags, and these four requirement sub-tags are connected by logical operators to form an infix expression, which indicates that the user expects the task to be executed to be executed on an execution node with a large memory and a neural network processing unit (NPU), and an operating system (OS) of centos or ubuntu.

[0071] As can be seen, the infix expression method is a logical expression similar to "AND OR NOT." Compared to expressing resource allocation requirements using subtag sets, expressing resource allocation requirements using "AND OR NOT" logical expressions can meet more user needs and improve the usability and functional completeness of the task scheduling system.

[0072] In an embodiment of the present application, a user may use a command line tool to submit a task, as shown in the following example.

[0073] / / Users submit jobs that need to be scheduled to execution nodes that have bigmem, an NPU, and an OS of centos or ubuntu.

[0074] job_cli–-labels'bigmem&npu&(centos|ubuntu)'myjob.sh

[0075] Of course, users can also submit tasks in other ways. For example, the task scheduling system provides users with a visual operation interface, namely a user interface, which displays multiple types of demand sub-tags for selection. Users can select a demand sub-tag on the user interface and, after selecting a demand sub-tag, click the Submit button on the user interface to submit the task.

[0076] For example, the user interface sequentially displays multiple types of requirement subtags, each type of requirement subtag having multiple subtags. For example, the operating system requirement subtags include centos, ubuntu, and so on. If the user selects only 'centos', it indicates that the user wishes to execute the task on an execution node whose OS is centos. If the user selects both 'centos' and 'ubuntu', it indicates that the user wishes to execute the task on an execution node whose OS is centos or ubuntu. In other words, if the user selects at least two requirement subtags of the same type, it indicates that the user wishes to execute the task on an execution node that matches any one of the at least two requirement subtags.

[0077] To simplify task tagging, the command line interface or visual interface displays tagging prompts. These prompts may include a specific example tag and / or logical operators such as AND, OR, NOT, and parentheses. Users can tag tasks based on these examples or select appropriate logical operators based on the prompts.

[0078] Step 402: confirm whether the historical resource allocation information recorded by the management node contains the same historical resource allocation requirements as the resource allocation requirements of the task to be executed. The historical resource allocation information is used to record the information of the execution node that matches the resource allocation requirements of the executed task.

[0079] Specifically, the management node first searches through its historical resource allocation information for historical resource allocation requirements that match the resource allocation requirements of the task to be executed. This historical resource allocation information records the execution nodes that the management node matched for the resource allocation requirements of the executed tasks. For example, this historical resource allocation information records the correspondence between requirement labels and matching execution nodes. The execution nodes that match the requirement representations recorded in the historical resource allocation information are referred to as candidate execution nodes.

[0080] Since the resource allocation requirements of the tasks to be executed have multiple possible representations, for example, when the resource allocation requirements are represented by requirement tags, requirement tags with the same semantics as a certain requirement tag may have multiple representations. In order to unify the representations for label matching, certain rules can be preset so that these multiple representations can be converted into a unique representation according to the preset rules.

[0081] For example, the first requirement tag of the task to be executed submitted by the user includes at least one requirement subtag, and the at least one requirement subtag may contain English capital letters. In this case, the at least one requirement subtag can be uniformly converted into English lowercase letters according to preset rules. For another example, the first requirement tag includes a subtag set formed by multiple requirement subtags. These multiple requirement subtags may not be arranged in dictionary order. In this case, these multiple requirement subtags can be rearranged in dictionary order according to preset rules. For another example, the first requirement tag is an infix expression including multiple requirement subtags. In this case, the infix expression can be converted into a postfix expression according to preset rules. The postfix expression does not contain English capital letters, and the multiple leaf nodes of each parent node in the tree structure corresponding to the postfix expression are arranged in dictionary order. The above rules can be used alone or in combination according to actual needs. Among them, the postfix expression is also called reverse Polish notation or reverse Polish notation, and reverse Polish notation is more conducive to computer understanding and calculation.

[0082] 5 is a schematic diagram of a tree structure represented by Reverse Polish Notation provided in an embodiment of the present application. The second requirement tag represented by Reverse Polish Notation is 'bigmem npu¢os ubuntu|&', and the corresponding first requirement tag is 'bigmem&npu&(centos|ubuntu)'.

[0083] It should be understood that if the requirement tag submitted by the user is used directly, for example, the requirement tag submitted by the user includes n requirement sub-tags, then the requirement tags with the same semantics as this requirement tag may reach up to n! -1. For example, the requirement tag submitted by the user is 'ab c', and there are 5 requirement tags with the same semantics as the requirement tag 'ab c', namely 'ac b', 'ba c', 'bc a', 'ca b', and 'cb a'. If the user also uses logical operators such as 'and, or, not, brackets' to represent the requirement tags, then the number of requirement tags with the same semantics as the requirement tags submitted by the user may be even greater. If a unified conversion is not performed, the amount of computation for tag matching will increase. Therefore, performing a unified tag conversion according to preset rules can greatly reduce the amount of computation and improve the cache hit rate.

[0084] Based on the above description, the management node converts the first requirement tag into a second requirement tag according to a preset rule. The second requirement tag has the same semantics as the first requirement tag but a different representation. If the second requirement tag is recorded in the historical resource allocation information, the management node determines that the historical resource allocation information contains a historical resource allocation requirement that is identical to the resource allocation requirement of the task to be executed.

[0085] For example, the first requirement tag is 'bigmem&NPU&(ubuntu|centos)'. The preset rule indicates that the first requirement tag is converted to a second requirement tag represented by a suffix expression, all uppercase letters in the first requirement tag are converted to lowercase, and the leaf nodes of each parent node in the tree structure corresponding to the suffix expression are arranged in lexicographic order. In this case, the second requirement tag is 'bigmem npu¢os ubuntu|&'.

[0086] In the following text, the first requirement label is expressed as an infix expression, and the second requirement label is expressed as a postfix expression.

[0087] In the embodiment of the present application, the demand tags recorded in the historical resource allocation information are all in the form of suffix expressions, and the corresponding demand tags are recorded according to the tree structure corresponding to the suffix expressions.

[0088] It should be understood that in the application embodiments, the historical resource allocation information records the correspondence between resource allocation requirements and candidate execution nodes. In one implementation, the correspondence includes a requirement tag and the identifier of the corresponding candidate execution node, where the identifier of the execution node can be the node name or other unique identifier of the execution node.

[0089] For example, the identifier of a certain execution node is "Node1".

[0090] In another implementation, the correspondence includes a requirement tag and a corresponding binary string, wherein the binary string includes a multi-digit binary number that corresponds one-to-one to multiple execution nodes in the task scheduling system, and the candidate execution nodes include the execution nodes corresponding to the positions in the corresponding binary string where the binary number has a first value. In the embodiment of the present application, the first value is '1'. In other embodiments, the first value may also be '0'.

[0091] For example, a binary string recorded in the historical resource allocation information is '100010'. These six binary digits correspond to six execution nodes in the task scheduling system. The candidate execution nodes represented by the binary string include the first execution node and the sixth execution node.

[0092] Optionally, historical resource allocation information is stored in the management node's cache. This improves scheduling efficiency. Specifically, in the event of a cache hit, the management node directly retrieves the target execution node from the cache. A cache hit is indicated by the historical resource allocation information containing the same resource allocation requirements as the task to be executed. Of course, historical resource allocation information can also be stored in other storage spaces on the management node or in the storage space of other devices.

[0093] Step 403: When it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, the execution node recorded in the historical resource allocation information and matching the historical resource allocation requirement is determined as the target execution node.

[0094] That is, if the historical resource allocation information records a historical resource allocation requirement that is identical to the resource allocation requirement of the task to be executed, the management node determines the first candidate execution node recorded in the historical resource allocation information as the target execution node. The first candidate execution node refers to the candidate execution node corresponding to the historical resource allocation requirement that is identical to the resource allocation requirement of the task to be executed. In the case where the resource allocation requirement of the task to be executed is represented by a first requirement tag, the first candidate execution node refers to the candidate execution node corresponding to the requirement tag (such as the second requirement tag mentioned above) recorded in the historical resource allocation information that has the same semantics as the first requirement tag.

[0095] If the management node determines that the historical resource allocation information contains the second requirement tag, it determines that the historical resource allocation information contains the same resource allocation requirement as the task to be executed, and determines the first candidate execution node as the target execution node. This eliminates the need to match candidate execution nodes from multiple execution nodes for each task to be executed, which improves scheduling efficiency.

[0096] Step 404: Schedule the task to be executed to the target execution node for execution.

[0097] Among them, if the target execution node includes one execution node, the management node will directly schedule the task to be executed to the execution node. If the target execution node includes at least two execution nodes, the management node determines at least one execution node from the target execution node according to the specified strategy, and schedules the task to be executed to the at least one execution node. Among them, the specified strategy is a random selection strategy, that is, the management node randomly selects at least one execution node from the target execution node. Alternatively, the specified strategy is a load balancing strategy, that is, the management node selects at least one execution node from the target execution node according to the load balancing strategy. Alternatively, the specified strategy can also be other strategies, which is not limited in the embodiments of the present application.

[0098] Furthermore, the number of the at least one execution node may be determined based on the amount of tasks to be executed or information indicating the number of nodes for the tasks to be executed obtained by the management node, or may be determined by the management node according to a default setting, or may be determined in other ways. These methods for determining the number of execution nodes may be used individually or in combination.

[0099] It should be understood that if a task to be executed needs to be executed by a single execution node, the management node will select an execution node from the target execution nodes and schedule the task to be executed to that execution node. If a task to be executed needs to be jointly executed by two execution nodes (i.e., cooperative execution), the management node will select two execution nodes from the target execution nodes and schedule the task to be executed to these two execution nodes. And so on.

[0100] In one implementation, the task information of the task to be executed also includes node quantity indication information. For example, the node quantity indication information is a first quantity, and the first quantity refers to the number of execution nodes required to execute the task to be executed. The management node selects a first number of execution nodes from the target execution nodes and schedules the task to be executed to the selected first number of execution nodes for execution.

[0101] In another implementation, the task information of the task to be executed also indicates the task amount. The larger the task amount, the more execution nodes are required. The management node selects an appropriate number of execution nodes from the target execution nodes according to the task amount of the task to be executed.

[0102] If the historical resource allocation information does not record the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, such as the second requirement tag is not recorded, that is, the candidate execution node corresponding to the requirement tag with the same semantics as the first requirement tag is not recorded, then the management node determines the execution node whose node tag matches the resource allocation requirements of the task to be executed from the multiple execution nodes, and determines the execution node whose node tag matches the resource allocation requirements of the task to be executed from the multiple execution nodes as the target execution node. That is, if the historical resource allocation information does not record the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, such as if a task with the same semantics as the first requirement tag has not been received in the past, then the management node needs to re-match the target execution node for the task to be executed from the multiple execution nodes, and cannot directly obtain the target execution node from the historical resource allocation information.

[0103] Among them, if the management node converts the first requirement label into the second requirement label according to the preset rules, if the second requirement label is not recorded in the historical resource allocation information, the management node determines the execution node whose node label matches the second requirement label from the multiple execution nodes, and determines the execution node whose node label matches the second requirement label among the multiple execution nodes as the target execution node.

[0104] It should be understood that, since the first requirement label and the second requirement label have the same semantics, the execution node whose node label matches the first requirement label among the multiple execution nodes is also the execution node that matches the second requirement label.

[0105] As can be seen from the foregoing, the second requirement tag includes at least one requirement subtag. There are many ways for the management node to determine the execution node whose node tag matches the second requirement tag from the multiple execution nodes. Here, one implementation method is described in detail.

[0106] One implementation method for the management node to determine the execution node whose node label matches the second requirement label from the multiple execution nodes is: obtaining the binary strings corresponding to the at least one requirement sub-label from the node bitmap, the node bitmap including the binary strings corresponding to the multiple reference labels, each binary string including a multi-bit binary number corresponding one-to-one to the multiple execution nodes in the task scheduling system, each binary number indicating whether the corresponding execution node has the corresponding reference label; replacing at least one requirement sub-label in the target suffix expression with the corresponding binary string to obtain the expression to be operated, the target suffix expression refers to the suffix expression used to represent the second requirement label; determining the value of the expression to be operated by bit operation to obtain the target binary string; determining the execution node with the corresponding reference label in the target binary string as the execution node whose node label matches the second requirement label among the multiple execution nodes.

[0107] In one example, the task scheduling system includes five execution nodes, and the node labels of the five execution nodes are as follows.

[0108] Node 0: bigmem, npu, ubuntu

[0109] Node 1: bitmem, centos

[0110] Node 2: npu, ubuntu

[0111] Node 3: bigmem

[0112] Node 4: centos

[0113] In this example, there are four reference labels: 'bigmem', 'npu', 'ubuntu', and 'centos'. The node bitmap formed by the five execution nodes includes four binary strings corresponding to these four reference labels: '11010', '10100', '01001', and '10100'. The relationship between the reference labels and the node bitmap is shown in Table 1.

[0114] Table 1

[0115] For example, if the first requirement tag is 'bigmem&NPU&(centos|ubuntu)' and the second requirement tag is 'bigmem npu¢os ubuntu|&', the target postfix expression is 11010 10100&01001 10100|&. Bitwise operations yield the value of the target postfix expression as 10000. Because the target postfix expression has the same semantics as the infix expression representing the first requirement tag, to facilitate understanding, the calculation result is interpreted here using the human-like method of infix expression calculation. Specifically, at least one requirement subtag in the infix expression is replaced with the corresponding binary string to obtain an expression with the same semantics as the expression to be calculated. Specifically, replacing bigmem&npu&(centos|ubuntu) with the expression 11010&10100&(01001|10100) yields the value 10000 through bitwise operations, which is the value of the expression to be calculated. That is to say, the obtained target binary string is '10000', and only the first bit in the target binary string has a value of '1', indicating that only the first bit has a corresponding reference label. Then the management node determines execution node No. 0 corresponding to the first bit as the execution node whose node label matches the second requirement label among the above five execution nodes, and determines execution node No. 0 as the target execution node.

[0116] In the above example, a position with a value of 1 in the node bitmap indicates that the corresponding execution node has a corresponding reference tag, and a position with a value of 0 indicates that the corresponding execution node does not have a corresponding reference tag.

[0117] It should be understood that the above examples are only used to explain the present solution and are not used to limit the present solution. In a specific implementation, reference labels may also include 'arm', 'x86', 'aarch64', etc.

[0118] As can be seen from the above examples, compared with the string comparison method, the embodiment of the present application can greatly reduce the amount of calculation through the tree-structured reverse Polish notation (i.e., suffix expression) and bit operations, thereby improving scheduling efficiency and task execution efficiency.

[0119] Of course, in some other embodiments, such as in some scenarios where efficiency requirements are not particularly high, or in some scenarios where the computing power of the management node is strong, the execution node whose node label matches the second requirement label can also be determined from the multiple execution nodes by string comparison.

[0120] In an embodiment of the present application, after determining the execution node whose node label matches the first requirement label among the multiple execution nodes as the target execution node, the correspondence between the first requirement label and the target execution node is added to the historical resource allocation information.

[0121] In an embodiment where the management node converts the first requirement tag into the second requirement tag, the management node adds a correspondence between the second requirement tag and the identifier of the target execution node in the historical resource allocation information, or adds a correspondence between the second requirement tag and the target binary string in the historical resource allocation information.

[0122] By recording a binary string in the historical resource allocation information to represent the corresponding candidate execution node, the amount of stored data can be reduced. When the historical resource allocation information is recorded in the cache of the management node, the waste of the cache can also be reduced.

[0123] It should be understood that the demand tag and binary string recorded in historical resource allocation information can be viewed as a hash key-value pair, where the demand tag serves as the 'key' and the binary string serves as the 'hash value'. By caching hash key-value pairs, cache hit rates can be greatly improved and memory usage can be reduced.

[0124] Optionally, after obtaining the target binary string, the management node directly adds the correspondence between the second requirement tag and the target binary string to the cached historical resource allocation information. Alternatively, after obtaining the target binary string, the management node determines whether the correspondence between the second requirement tag and the target binary string meets the cache entry condition. If so, the management node adds the correspondence between the second requirement tag and the target binary string to the cached historical resource allocation information. If not, the management node does not add the correspondence between the second requirement tag and the target binary string to the cached historical resource allocation information.

[0125] The cache entry conditions described above may include a historical resource allocation demand recorded in the historical resource allocation information including a resource allocation demand with a number of sub-demands exceeding a first threshold, and / or a resource allocation demand with a frequency exceeding a second threshold. Other conditions may also be included. Thus, if the number of demand sub-tags included in the second demand tag does not exceed the first threshold, and / or the frequency of demands corresponding to the demand sub-tags included in the second demand tag exceeds the second threshold, the management node adds the correspondence between the second demand tag and the target binary string to the historical resource allocation information.

[0126] Among them, the first threshold value can be 3 or 5 or other values. The smaller the first threshold value is, the relatively fewer requirements the corresponding task has on the execution node, and users usually do not set many demand sub-tags. In other words, the fewer demand sub-tags a task has, the closer the task is to the popular demand task, and the cache stores as many candidate execution nodes that match the popular demand tasks as possible, thereby providing a cache hit rate. In other words, the cache hit rate can be improved by limiting the first threshold value. Similarly, the second threshold value can be a value such as 10 or 15. The larger the second threshold value is, the higher the corresponding resource demand is, and information with high cache demand can improve the cache hit rate.

[0127] In addition, since the cache space is limited, the management node may eliminate outdated cache in the historical resource allocation information according to the least recently used (LRU) principle or other principles, so as to provide cache space for the newly added corresponding relationship.

[0128] In an embodiment of the present application, before using the above-mentioned node bitmap to determine the expression to be operated, the management node first generates a node bitmap based on the node labels of each execution node in the task scheduling system and multiple reference labels. The generated node bitmap represents whether each execution node in the multiple execution nodes has each reference label in the multiple reference labels.

[0129] Before generating the node bitmap, the management node first obtains the node label of each execution node among the multiple execution nodes. There are multiple ways to obtain the node label, such as the administrator manually labeling the execution node and / or the system automatically labeling the execution node.

[0130] In the method where the administrator manually labels the execution nodes, the administrator configures node labels for one or at least two execution nodes through a command line tool or a visual operation interface. One or more node sub-labels can be configured for each execution node, and the configuration information is submitted to the management node. For example, the administrator configures at least two node sub-labels for the first execution node among the multiple execution nodes, generates first configuration information through a command line tool or a visual operation interface, and submits the first configuration information to the management node. For another example, the administrator configures node labels for at least two execution nodes among the multiple execution nodes and submits second configuration information to the management node.

[0131] The following is an example of how an administrator uses the command line tool to label an execution node.

[0132] / / Label node1 and node3 with bigmem and npu

[0133] node_cli–-lables'bigmem,npu'node1node3

[0134] / / Add gpu labels to node1, node2 and node4

[0135] node_cli-l'gpu'node1node2node4

[0136] Accordingly, the management node receives first configuration information, the first configuration information includes at least two node sub-labels configured for a first execution node among the multiple execution nodes, and the first execution node is any execution node whose node label includes multiple node sub-labels; and / or, the management node receives second configuration information, the second configuration information includes node labels configured for at least two execution nodes among the multiple execution nodes.

[0137] As can be seen, this solution supports configuring multiple node sub-labels for a single execution node at once, as well as configuring node labels for a batch of execution nodes at once. This increases administrators' flexibility in configuring custom labels and improves the usability and functionality of the task scheduling system.

[0138] In the method where the system automatically labels the execution node, the execution node actively reports its own node information, or sends the node information of the execution node to the management node after receiving the request of the management node. The management node automatically obtains information that can be used as the node sub-label from the node information.

[0139] For example, the node information reported by the first execution node includes 'OS is centos, CPU architecture is x86_64', then the management node may automatically set a node label including 'centos, x86_64' for the first execution node.

[0140] It should be understood that in order to ensure that the custom labels configured by the administrator do not conflict with the labels automatically added by the system, when the first node sub-label in the configuration information received by the management node conflicts with the node sub-label automatically added by the system, the management node sends a prompt message to the client used by the administrator, indicating that the first node sub-label configured by the administrator conflicts with the node sub-label automatically added by the system.

[0141] Next, please refer to Figure 6 for a further exemplary explanation of the implementation process of the management node obtaining the node label and generating the node bitmap in the embodiment of the present application.

[0142] Figure 6 is a flow chart of a method for obtaining node labels and generating node bitmaps provided by an embodiment of the present application. Taking the management node including the resource management interface, resource management module and job scheduling module shown in Figure 2 as an example, see Figure 6, the resource management module receives the node label marked by the administrator through the management interface (i.e., receives configuration information), and receives the node label reported by the execution node (i.e., the system automatically marks the label), and constructs a binary string corresponding to the corresponding reference label based on the node label of the execution node, i.e., constructs a bitmap of the corresponding reference label, and constructs a global node bitmap based on the bitmap of all reference labels. The resource management module synchronizes the node bitmap to the job scheduling module, i.e., the node bitmap is synchronized to the scheduling engine (i.e., the job scheduling module), so that the job scheduling module can match the target execution node for the task to be executed based on the node bitmap.

[0143] Next, please refer to Figure 7 to exemplify an implementation method of the task scheduling method provided in an embodiment of the present application.

[0144] FIG7 is a flowchart of another task scheduling method provided by an embodiment of the present application. Referring to FIG7 , the management node receives a task submitted by a user through a job submission interface, and the task information of the task includes a requirement tag represented by an infix expression. The management node parses the infix expression through the resource management module, constructs a reverse Polish notation tree structure corresponding to the infix expression, and sends the task to the job scheduling module in the management node, that is, sends the task to the scheduling engine, including sending the constructed tree structure.

[0145] The job scheduling module selects a task from the tasks to be executed for scheduling. The job scheduling module determines to enter the label requirement filtering phase. In this phase, the job scheduling module queries the historical resource allocation information cached by the management node to see whether there are candidate execution nodes corresponding to the requirement tag with the same semantics as the requirement tag of the task to be executed. Simply put, in this phase, the cache is first queried. If the cache hits, the job scheduling module directly obtains the candidate execution node corresponding to the requirement tag with the same semantics as the requirement tag of the task to be executed, and obtains the target execution node, that is, directly obtains the candidate execution node that meets the label requirement.

[0146] If the cache misses, the job scheduling module determines the target binary string through bit operations based on the node bitmap and the above-mentioned reverse Polish notation, that is, calculates the candidate execution node that meets the label requirements. For the specific implementation process, please refer to the relevant content above, which will not be repeated here. After determining the target binary string, the job scheduling module determines the target execution node based on the target binary string. For the specific implementation process, please refer to the relevant content above. At this point, the job scheduling module calculates the node set that meets the demand label requirements. The execution node in this node set is the target execution node, and the target execution node can include one execution node or at least two execution nodes.

[0147] After determining the target execution node, the job scheduling module performs subsequent scheduling processes based on the target execution node. For example, it determines at least one execution node from the target execution nodes according to a specified strategy and schedules the task to be executed to the at least one execution node.

[0148] Furthermore, the job scheduling module determines whether the correspondence between the reverse Polish notation and the target binary string satisfies a cache entry condition. If so, the job scheduling module adds the correspondence between the reverse Polish notation and the target binary string to the historical resource allocation information, i.e., caches the correspondence. Furthermore, due to limited cache space, the job scheduling module may eliminate outdated cached data in the historical resource allocation information according to the LRU principle to make room for the newly added correspondence.

[0149] In summary, in an embodiment of the present application, after obtaining the resource allocation requirements of a pending task, if the historical resource allocation information contains a historical resource allocation requirement that is identical to the resource allocation requirement of the pending task, the execution node that matches the historical resource allocation requirement recorded in the historical resource allocation information is directly determined as the target execution node. This eliminates the need to re-identify the execution node whose node label matches the resource allocation requirement of the pending task from multiple execution nodes, eliminates the need for extensive string matching calculations, and improves the efficiency of executing node screening, thereby improving scheduling efficiency and task execution efficiency.

[0150] In its implementation, users can describe task resource allocation requirements using AND, OR, and NOT logical syntax. Administrators can customize labels for execution nodes, assign multiple node sublabels to a single execution node at a time, and label multiple execution nodes sequentially (i.e., set operations). The management node uses bitwise operations and Reverse Polish notation to convert label scheduling into efficient bitwise operations, improving performance. Hash key-value pairs are constructed to cache scheduling results, and Reverse Polish notation is used to merge label requirements with the same semantics, improving cache hit rates.

[0151] FIG8 is a schematic diagram of the structure of a task scheduling device 800 provided in an embodiment of the present application. The task scheduling device 800 can be implemented as part or all of a computer device by software, hardware, or a combination of both. The computer device can be the management node shown in FIG1 or FIG2. In an embodiment of the present application, the device 800 is included in the management node of the task scheduling system, and the task scheduling system also includes multiple execution nodes. Referring to FIG8, the device 800 includes: a first acquisition module 801, a first determination module 802, a second determination module 803, and a scheduling module 804.

[0152] The first acquisition module 801 is used to obtain the resource allocation requirements of the task to be executed;

[0153] A first determining module 802 is configured to determine whether the historical resource allocation information recorded by the management node contains the same historical resource allocation requirements as the resource allocation requirements of the task to be executed. The historical resource allocation information is used to record information about the execution node for which the management node matches the resource allocation requirements of the executed task.

[0154] The second determining module 803 is configured to, when it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, determine the execution node that matches the historical resource allocation requirement recorded in the historical resource allocation information as the target execution node;

[0155] The scheduling module 804 is used to schedule the task to be executed to the target execution node for execution.

[0156] Optionally, the resource allocation requirement includes multiple sub-requirements, and the multiple sub-requirements are expressed in the form of logical expressions.

[0157] Optionally, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold.

[0158] Optionally, the historical resource allocation demands recorded in the historical resource allocation information include resource allocation demands whose number of demands exceeds a second threshold.

[0159] Optionally, the resource allocation requirement of the task to be executed is represented by a first requirement tag; the apparatus 800 further includes:

[0160] a label conversion module, configured to convert the first requirement label into a second requirement label according to a preset rule, wherein the second requirement label and the first requirement label have the same semantics but different representation forms;

[0161] The first determining module is specifically configured to determine whether the historical resource allocation information records a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed, if the historical resource allocation information records a second requirement tag.

[0162] Optionally, the apparatus 800 further includes:

[0163] a demand matching module, configured to determine, from a plurality of execution nodes, an execution node whose node label matches the second demand label if the second demand label is not recorded in the historical resource allocation information;

[0164] The third determining module is configured to determine an execution node whose node label matches the second requirement label among the multiple execution nodes as a target execution node.

[0165] Optionally, the first requirement tag is expressed in the form of an infix expression, and the second requirement tag is expressed in the form of a postfix expression.

[0166] Optionally, the second requirement tag includes at least one requirement subtag;

[0167] The demand matching module is specifically used to:

[0168] Obtaining binary strings corresponding to the at least one requirement sub-tag from a node bitmap, the node bitmap including binary strings corresponding to a plurality of reference tags, each binary string including a plurality of binary digits corresponding one-to-one to the plurality of execution nodes, each binary digit indicating whether the corresponding execution node has a corresponding reference tag;

[0169] Replacing the at least one requirement subtag in the target suffix expression with a corresponding binary string to obtain an expression to be operated, where the target suffix expression refers to a suffix expression used to represent the second requirement tag;

[0170] Determine the value of the expression to be operated through bit operations to obtain the target binary string;

[0171] An execution node having a corresponding reference label in the target binary string is determined as an execution node whose node label matches the second requirement label among the multiple execution nodes.

[0172] Optionally, the apparatus 800 further includes:

[0173] The adding module is used to add a corresponding relationship between the second requirement label and the target binary string in the historical resource allocation information.

[0174] Optionally, the apparatus 800 further includes:

[0175] A second acquisition module is used to obtain a node label of each execution node in the multiple execution nodes;

[0176] A generation module is used to generate a node bitmap based on the node label of each execution node in the multiple execution nodes and the multiple reference labels, where the node bitmap represents whether each execution node in the multiple execution nodes has each reference label in the multiple reference labels.

[0177] Optionally, the second acquisition module is specifically configured to:

[0178] Receive first configuration information, where the first configuration information includes at least two node sub-tags configured for a first execution node among the multiple execution nodes, where the first execution node is any execution node whose node tag includes the multiple node sub-tags; and / or,

[0179] Second configuration information is received, where the second configuration information includes node labels configured for at least two execution nodes among the plurality of execution nodes.

[0180] Optionally, historical resource allocation information is recorded in a cache of the management node.

[0181] In an embodiment of the present application, after obtaining the resource allocation requirements of a pending task, if the historical resource allocation information contains a historical resource allocation requirement that is identical to the resource allocation requirement of the pending task, the execution node that matches the historical resource allocation requirement recorded in the historical resource allocation information is directly determined as the target execution node. This eliminates the need to re-identify the execution node whose node label matches the resource allocation requirement of the pending task from multiple execution nodes, eliminates the need for extensive string matching calculations, and improves the efficiency of executing node screening, thereby improving scheduling efficiency and task execution efficiency.

[0182] It should be noted that the task scheduling device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate task scheduling. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the task scheduling device provided in the above embodiment and the task scheduling method embodiment are of the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0183] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0184] It should be understood that the "at least one" mentioned herein refers to one or more, and "a plurality of" refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0185] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0186] The above description is an embodiment provided for this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A task scheduling method, characterized in that: The method is applied to a management node in a task scheduling system, wherein the task scheduling system further includes a plurality of execution nodes. Obtain resource allocation requirements for tasks to be performed; Confirm whether there is a historical resource allocation requirement in the historical resource allocation information recorded by the management node that is the same as the resource allocation requirement of the task to be executed, the historical resource allocation information is used to record the information of the execution node that the management node matches the resource allocation requirement of the executed task; When it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, determining an execution node that matches the historical resource allocation requirement and is recorded in the historical resource allocation information as a target execution node; The task to be executed is dispatched to the target execution node for execution.

2. The method according to claim 1, characterized in that The resource allocation requirement includes multiple sub-requirements, and the multiple sub-requirements are expressed in the form of logical expressions.

3. The method according to claim 1 or 2, characterized in that The historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold.

4. The method according to any one of claims 1 to 3, characterized in that: The historical resource allocation demands recorded in the historical resource allocation information include resource allocation demands whose demand times exceed a second threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The resource allocation requirement of the task to be executed is represented by a first requirement tag; the method further comprises: Converting the first requirement tag into a second requirement tag according to a preset rule, wherein the second requirement tag and the first requirement tag have the same semantics but different representation forms; If the second requirement tag is recorded in the historical resource allocation information, it is determined that the historical resource allocation information records a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed.

6. The method according to claim 5, characterized in that The method further comprises: If the second requirement tag is not recorded in the historical resource allocation information, determining an execution node whose node tag matches the second requirement tag from the multiple execution nodes; An execution node among the multiple execution nodes whose node label matches the second requirement label is determined as the target execution node.

7. The method according to claim 5 or 6, characterized in that The first requirement tag is expressed in the form of an infix expression, and the second requirement tag is expressed in the form of a postfix expression.

8. The method according to claim 6, characterized in that The second requirement tag includes at least one requirement sub-tag; The determining, from the multiple execution nodes, an execution node whose node label matches the second requirement label includes: Obtaining binary strings corresponding to the at least one requirement sub-tag from a node bitmap, wherein the node bitmap includes binary strings corresponding to multiple reference tags, each binary string includes multiple binary digits corresponding to the multiple execution nodes one by one, and each binary digit indicates whether the corresponding execution node has a corresponding reference tag; Replacing the at least one requirement sub-tag in the target suffix expression with a corresponding binary string to obtain an expression to be operated, wherein the target suffix expression refers to a suffix expression used to represent the second requirement tag; Determine the value of the expression to be operated by bit operation to obtain a target binary string; An execution node having a corresponding reference label in the target binary string is determined as an execution node whose node label matches the second requirement label among the multiple execution nodes.

9. The method according to claim 8, characterized in that The method further comprises: A correspondence between the second requirement tag and the target binary character string is added to the historical resource allocation information.

10. The method according to claim 8 or 9, characterized in that The method further comprises: Obtaining a node label of each execution node among the multiple execution nodes; The node bitmap is generated based on the node label of each execution node among the multiple execution nodes and the multiple reference labels, and the node bitmap indicates whether each execution node among the multiple execution nodes has each reference label among the multiple reference labels.

11. The method according to claim 10, characterized in that The obtaining of the node label of each execution node among the plurality of execution nodes comprises: receiving first configuration information, wherein the first configuration information includes at least two node sub-labels configured for a first execution node among the plurality of execution nodes, the first execution node being any execution node whose node label includes a plurality of node sub-labels; and / or, Second configuration information is received, where the second configuration information includes node labels configured for at least two execution nodes among the plurality of execution nodes.

12. The method according to any one of claims 1 to 11, characterized in that: The historical resource allocation information is recorded in the cache of the management node.

13. A task scheduling device, characterized in that: A management node included in a task scheduling system, the task scheduling system also includes a plurality of execution nodes, the device includes: A first acquisition module is used to acquire resource allocation requirements of tasks to be executed; A first determination module is used to confirm whether there is a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed in the historical resource allocation information recorded by the management node, and the historical resource allocation information is used to record the information of the execution node that the management node matches the resource allocation requirement of the executed task; A second determination module is used to determine, when it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, an execution node recorded in the historical resource allocation information and matching the historical resource allocation requirement as a target execution node; The scheduling module is used to schedule the task to be executed to the target execution node.

14. The device according to claim 13, characterized in that The historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold.

15. The device according to claim 13 or 14, characterized in that The resource allocation requirement of the task to be executed is represented by a first requirement tag; the device also includes: A label conversion module, used to convert the first requirement label into a second requirement label according to a preset rule, wherein the second requirement label and the first requirement label have the same semantics but different representation forms; The first determining module is specifically configured to determine that a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed is recorded in the historical resource allocation information if the second requirement tag is recorded in the historical resource allocation information.

16. The device according to claim 15, characterized in that The device also includes: a demand matching module, configured to determine, from the plurality of execution nodes, an execution node whose node label matches the second demand label if the second demand label is not recorded in the historical resource allocation information; The third determination module is used to determine an execution node whose node label matches the second requirement label among the multiple execution nodes as the target execution node.

17. The device according to claim 15 or 16, characterized in that The first requirement tag is expressed in the form of an infix expression, and the second requirement tag is expressed in the form of a postfix expression.

18. A task scheduling system, characterized in that: The task scheduling system includes a management node and multiple execution nodes; The management node is used to execute the steps of the method according to any one of claims 1 to 12.

19. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

20. A computer program product, characterized in that The computer program product stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images