Task scheduling method, device and system, storage medium and computer program product

Abstract

The invention discloses a task scheduling method, device and system, a storage medium and a computer program product, and belongs to the technical field of computers. The method comprises the following steps: after a resource allocation demand of a to-be-executed task is obtained, if a historical resource allocation demand which is the same as the resource allocation demand of the to-be-executed task is recorded in historical resource allocation information, executing the to-be-executed task; and if yes, directly determining the execution node which is recorded in the historical resource allocation information and matched with the historical resource allocation demand as the target execution node. Therefore, the execution node of which the node label is matched with the resource allocation demand of the to-be-executed task does not need to be re-determined from the plurality of execution nodes, a large amount of character string matching calculation is not needed, the screening efficiency of the execution nodes is improved, and the scheduling efficiency and the task execution efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task scheduling method, device, system, storage medium and computer program product. Background Art

[0002] Currently, users can encapsulate business logic into tasks through the client and submit relevant information of the tasks to the management node in the task scheduling system. The task scheduling system also includes multiple execution nodes. The management node is responsible for scheduling each task submitted by the user to at least one of the multiple execution nodes according to the scheduling strategy, so that the at least one execution node executes the task. A reasonable and efficient scheduling method can improve the resource utilization of the task scheduling system and improve the execution efficiency of tasks.

[0003] With the development of scheduling technology, scheduling technology based on node labels has emerged. In the related technology of task scheduling based on node labels, each execution node has a node label, and the node label represents one or more information of the central processing unit (CPU) architecture, operating system type, whether it includes heterogeneous processors, whether it includes graphics processing units (GPU), etc. of the corresponding execution node. Users can specify the requirement label when encapsulating tasks, so that the relevant information of the task submitted by the user includes the requirement label. After receiving the task with the requirement label, the management node matches the requirement label with the node label of each execution node in the multiple execution nodes by string comparison, thereby filtering out the execution node matching the task, and then scheduling the task to the filtered execution node. For example, the requirement label of the task submitted by the user includes 'x86', indicating that the user wants to execute the task on the execution node with the CPU architecture of the x86 architecture. Then, the management node first filters out all the execution nodes with the CPU architecture of the x86 architecture in the task scheduling system by string comparison, and then filters out the execution node for executing the task from all the execution nodes with the CPU architecture of the x86 architecture.

[0004] However, the number of tasks that the task scheduling system needs to process is huge, and the number of execution nodes in the task scheduling system is also relatively large. The above method needs to compare the requirement label of each task with the node label of each execution node, resulting in low screening efficiency and affecting scheduling efficiency. In addition, the calculation amount of screening execution nodes by string comparison is large, which will also affect scheduling efficiency and thus affect task execution efficiency. Summary of the invention

[0005] The present application provides a task scheduling method, device, storage medium and computer program product, which can improve scheduling efficiency and task execution efficiency. The technical solution is as follows:

[0006] In a first aspect, a task scheduling method is provided, which is applied to a management node in a task scheduling system, wherein the task scheduling system further includes a plurality of execution nodes, and the method includes:

[0007] Obtain the resource allocation requirements of the task to be executed; confirm whether there is any historical resource allocation requirement in the historical resource allocation information recorded by the management node that is the same as the resource allocation requirement of the task to be executed, the historical resource allocation information is used to record the information of the execution node that matches the resource allocation requirement of the executed task by the management node; when it is determined that there is any historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, determine the execution node that matches the historical resource allocation requirement recorded in the historical resource allocation information as the target execution node; and schedule the task to be executed to the target execution node for execution.

[0008] That is, if the historical resource allocation information records historical resource allocation requirements that are the same as the resource allocation requirements of the task to be executed, the execution node recorded in the historical resource allocation information that matches the historical resource allocation requirements is directly determined as the target execution node. In this way, there is no need to re-determine the execution node whose node label matches the resource allocation requirement of the task to be executed from multiple execution nodes, and there is no need for a large amount of string matching calculations, which improves the screening efficiency of the execution nodes, thereby improving the scheduling efficiency and the task execution efficiency.

[0009] Optionally, the resource allocation requirement includes multiple sub-requirements, and the multiple sub-requirements are expressed in the form of logical expressions. Compared with expressing resource allocation requirements in the form of sub-tag sets, expressing resource allocation requirements in the form of logical expressions can meet more user needs and improve the usability and functional completeness of the task scheduling system.

[0010] Optionally, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold. And / or, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of requirements exceeds a second threshold. In this way, the hit rate of obtaining the target execution node from the historical resource allocation information can be improved.

[0011] Optionally, the resource allocation requirement of the task to be executed is represented by a first requirement tag, and the first requirement tag is represented in the form of an infix expression. The infix expression is a logical expression of "AND or NOT". Compared with the sub-tag set method to represent the requirement tag, the logical expression of "AND or NOT" to represent the requirement tag can meet more user needs and improve the usability and functional completeness of the task scheduling system.

[0012] Since a requirement tag with the same semantics as a requirement tag may have multiple representations, in order to unify the representations for label matching, certain rules can be preset so that these multiple representations can be converted into a unique representation according to the preset rules.

[0013] Optionally, the method further includes: converting the first requirement tag into a second requirement tag according to a preset rule, the second requirement tag and the first requirement tag being requirement tags with the same semantics but different representation forms; if the second requirement tag is recorded in the historical resource allocation information, determining that the historical resource allocation information records the same historical resource allocation requirement as the resource allocation requirement of the task to be executed. Performing unified tag conversion according to the preset rules can greatly reduce the amount of computation and improve the cache hit rate.

[0014] When the first requirement label is expressed in the form of an infix expression, the second requirement label is expressed in the form of a postfix expression. The postfix expression is also called reverse Polish notation or reverse Polish notation, which is more conducive to computer understanding and calculation.

[0015] Optionally, the method further includes: if the second requirement tag is not recorded in the historical resource allocation information, determining an execution node whose node tag matches the second requirement tag from the multiple execution nodes; and determining an execution node whose node tag matches the second requirement tag from the multiple execution nodes as a target execution node.

[0016] The second requirement tag includes at least one requirement sub-tag; determining the execution node whose node tag matches the second requirement tag from the multiple execution nodes includes: obtaining a binary string corresponding to at least one requirement sub-tag from a node bitmap, the node bitmap includes binary strings corresponding to multiple reference tags, each binary string includes a multi-bit binary number corresponding to the multiple execution nodes, each binary number indicates whether the corresponding execution node has a corresponding reference tag; replacing at least one requirement sub-tag in the target suffix expression with a corresponding binary string to obtain an expression to be operated, the target suffix expression refers to a suffix expression used to represent the second requirement tag; determining the value of the expression to be operated by bit operation to obtain a target binary string; determining the execution node with the corresponding reference tag in the target binary string as the execution node whose node tag matches the second requirement tag among the multiple execution nodes. The use of bit operations and reverse Polish notation can convert label scheduling into efficient bit operations to improve performance.

[0017] Optionally, a corresponding relationship between the second requirement tag and the target binary string is added to the historical resource allocation information. By recording the binary string in the historical resource allocation information to represent the corresponding execution node, the amount of stored data can be reduced.

[0018] Optionally, the historical resource allocation information is recorded in a cache of the management node. The scheduling efficiency can be improved by using a cache.

[0019] Optionally, the method also includes: obtaining a node label of each execution node among the multiple execution nodes; generating a node bitmap based on the node label of each execution node among the multiple execution nodes and multiple reference labels, wherein the node bitmap represents whether each execution node among the multiple execution nodes has each reference label among the multiple reference labels.

[0020] Wherein, obtaining the node label of each execution node among the multiple execution nodes includes: receiving first configuration information, the first configuration information includes at least two node sub-labels configured for the first execution node among the multiple execution nodes, the first execution node is any execution node whose node label includes the multiple node sub-labels; and / or, receiving second configuration information, the second configuration information includes node labels configured for at least two execution nodes among the multiple execution nodes. This solution supports configuring multiple node sub-labels for one execution node at a time, and also supports configuring node labels for a batch of execution nodes at a time. In this way, the flexibility of administrators in configuring custom labels can be improved, and the usability and functional completeness of the task scheduling system can be improved.

[0021] Optionally, scheduling the task to be executed to the target execution node for execution includes: determining at least one first target execution node from the target execution nodes, and scheduling the task to be executed to the at least one first target execution node for execution. That is, the target execution node serves as a candidate execution node, and the management node further screens out at least one execution node from the candidate execution nodes to ultimately execute the task. Among them, further screening strategies may include load balancing strategies or random selection strategies, etc. The number of the at least one execution node can be determined according to the task amount of the task to be executed or the node number indication information of the task to be executed, or it can also be determined by the management node according to the default settings. These methods for determining the number of execution nodes can be used alone or in combination.

[0022] In a second aspect, a task scheduling device is provided, which has the function of implementing the task scheduling method in the first aspect. The task scheduling device includes one or more modules, which are used to implement the task scheduling method provided in the first aspect.

[0023] In a third aspect, a task scheduling system is provided, which includes a management node and a plurality of execution nodes. The management node is used to execute the steps of the task scheduling method provided in the first aspect. The plurality of execution nodes are used to execute the tasks assigned by the management node.

[0024] In a fourth aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory being used to store a program for executing the task scheduling method provided in the first aspect, and storing data involved in implementing the task scheduling method provided in the first aspect. The processor is configured to execute the program stored in the memory. The computer device may also include a communication bus, the communication bus being used to establish a connection between the processor and the memory.

[0025] In a fifth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the task scheduling method described in the first aspect.

[0026] In a sixth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the task scheduling method described in the first aspect.

[0027] The technical effects obtained in the above-mentioned second to sixth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1It is a system architecture diagram involved in a task scheduling method provided in an embodiment of the present application;

[0029] Figure 2 is a system architecture diagram involved in another task scheduling method provided in an embodiment of the present application;

[0030] Figure 3 is a structural schematic diagram of a computer device provided in an embodiment of the present application;

[0031] Figure 4 is a flowchart of a task scheduling method provided by an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of a reverse Polish notation represented by a tree structure provided in an embodiment of the present application;

[0033] Figure 6 This is a flow chart of a method for obtaining node labels and generating node bitmaps provided in an embodiment of the present application;

[0034] Figure 7 is a flowchart of another task scheduling method provided in an embodiment of the present application;

[0035] Figure 8 It is a structural diagram of a task scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0037] Figure 1 is a system architecture diagram of a task scheduling method provided in an embodiment of the present application. The system may be referred to as a task scheduling system, or simply a scheduling system. Figure 1 The system includes a management node 101 and a plurality of execution nodes 102. The management node 101 and the plurality of execution nodes 102 are connected by wireless or wired communication.

[0038] The management node 101 is used to obtain task information of the task to be executed, the task information includes resource allocation requirements, and use this solution to determine the target execution node 102 for the task to be executed, and schedule the task to be executed to the target execution node 102 for execution. Optionally, the management node 101 is used to schedule the task to be executed to at least one execution node 102 among the target execution nodes 102 for execution.

[0039] Each execution node 102 among the multiple execution nodes 102 is used to execute the task assigned by the management node 101 .

[0040] In an embodiment of the present application, the multiple execution nodes 102 may be multiple computing nodes, multiple network nodes, multiple computing power network elements, multiple grids, or multiple computing processes. Among them, network nodes may include network devices such as switches and routers. Computing power network elements and grids can be understood as upper-level concepts of computing nodes. For example, a computing power network element may include multiple computing nodes. The computing process may be a Linux process. That is, this solution can be applied to various scheduling scenarios, including various distributed cluster computing scenarios, such as high performance computing (HPC) clusters, artificial intelligence (AI), big data, cloud services, microservice scheduling, network switching and other scenarios.

[0041] Next, please combine Figure 2 The modules included in the management node and the modules included in the execution node are introduced.

[0042] Figure 2 is a system architecture diagram of another task scheduling method provided in an embodiment of the present application. The system includes a management node and multiple execution nodes. Figure 2 Any one of the multiple execution nodes is shown in FIG.

[0043] See also Figure 2 The management node includes an application processing interface (API) interaction module, a resource management module, and a job scheduling module. In the embodiment of the present application, a job may also be referred to as a task in some scenarios. The API interaction module includes a resource management interface and a job submission interface. The job scheduling module may also be referred to as a scheduling engine, a task scheduling module, etc.

[0044] The resource management interface is used by the administrator, and the administrator can configure the node label for the execution node through the resource management interface, that is, the resource management interface is used to receive configuration information, and the configuration information includes the node label configured by the administrator for the execution node. Optionally, the administrator can also view, delete, and modify the node label of the execution node through the resource management interface, that is, the resource management interface is also used to receive node label query requests and / or node label update requests, and return corresponding information.

[0045] The job submission interface is for users to use. Users can submit tasks through the job submission interface. The task information of the submitted task may include resource allocation requirements, and the resource allocation requirements may be represented by a requirement tag. That is, the job submission interface is used to receive the task information of the task, and the task information includes the requirement tag.

[0046] The resource management module is used to obtain the node label of each execution node in the multiple execution nodes, including obtaining the configuration information submitted by the administrator, and optionally, also includes the node information of the execution nodes automatically reported by the execution nodes or automatically collected by the resource management module, and obtains a part of the node sub-labels included in the node label from the node information. The resource management module is also used to generate a node bitmap based on the node label of each execution node in the multiple execution nodes and multiple reference labels, and synchronize the node bitmap to the job scheduling module. Among them, the node bitmap represents whether each execution node in the multiple execution nodes has each reference label in the multiple reference labels. For specific implementation methods, please refer to the detailed description of the method embodiment below.

[0047] The job scheduling module is used to schedule tasks using this solution based on the above node bitmap, including determining the target execution node for the task to be executed, and scheduling the task to be executed to at least one execution node among the target execution nodes, that is, sending the task to the execution node.

[0048] The execution node includes a job control module and an operating system. The job control module is used to receive tasks issued by the management node, specifically, receive tasks issued by the job scheduling module, and run the received tasks in the operating system, that is, execute the corresponding tasks through the operating system.

[0049] Optionally, the execution node also includes a resource monitoring module. The resource monitoring module is used to collect node information of the execution node from the operating system, and the node information includes but is not limited to the following information: node name, performance information, operating system information, processor information, etc., wherein the operating system information and processor information can be used as node sub-labels. The resource monitoring module is also used to actively report the node information of the execution node to the management node, or send the node information of the execution node to the management node after receiving the request of the management node. As an example, the execution node sends the node information to the management node when registering with the task scheduling system and / or when going online.

[0050] Optionally, the resource monitoring module collects node information from the operating system as original information, and the resource monitoring system processes the original information and reports it to the management node, for example, unifying the format of the original information and encapsulating it into a message format.

[0051] The present application also provides a scheduler, see Figure 2 The scheduler includes a management side and a computing side. The management side includes the resource management interface, job submission interface, resource management module, and job scheduling module mentioned above. The computing side includes the resource monitoring module and job control module mentioned above. The scheduler can be regarded as the brain of the distributed computing cluster, which is mainly responsible for allocating corresponding system resources to the tasks submitted by users. Figure 2It can be seen that the scheduler provides rich functions and interfaces, which are convenient for users to use and administrators to operate and maintain, and can improve the utilization of system resources, improve scheduling efficiency and task execution efficiency.

[0052] It should be understood that the system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0053] Please refer to Figure 3 , Figure 3 is a schematic diagram of a computer device according to an embodiment of the present application. Optionally, the computer device is Figure 1 The management node or execution node shown in FIG. 1 includes one or more processors 301 , a communication bus 302 , a memory 303 and one or more communication interfaces 304 .

[0054] The processor 301 is a general-purpose central processing unit (CPU), a network processing (NP), a microprocessor, or one or more integrated circuits for implementing the solution of the present application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. Optionally, the PLD is a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0055] The communication bus 302 is used to transmit information between the above components. Optionally, the communication bus 302 is divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0056] Optionally, the memory 303 is a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 303 exists independently and is connected to the processor 301 through the communication bus 302, or the memory 303 is integrated with the processor 301.

[0057] The communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks. The communication interface 304 includes a wired communication interface, and optionally, also includes a wireless communication interface. Among them, the wired communication interface is, for example, an Ethernet interface. Optionally, the Ethernet interface is an optical interface, an electrical interface, or a combination thereof. The wireless communication interface is a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0058] Optionally, in some embodiments, the computer device includes multiple processors, such as Figure 3 The processor 301 and the processor 305 shown in FIG. Each of these processors is a single-core processor or a multi-core processor. Optionally, the processor here refers to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0059] In a specific implementation, as an embodiment, the computer device also includes an output device 306 and an input device 307. The output device 306 communicates with the processor 301 and can display information in a variety of ways. For example, the output device 306 is a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device or a projector. The input device 307 communicates with the processor 301 and can receive user input in a variety of ways. For example, the input device 307 is a mouse, a keyboard, a touch screen device or a sensor device.

[0060] In some embodiments, the memory 303 is used to store the program code 310 for executing the solution of the present application, and the processor 301 can execute the program code 310 stored in the memory 303. The program code includes one or more software modules, and the computer device can implement the following through the processor 301 and the program code 310 in the memory 303. Figure 4 The task scheduling method provided by the embodiment.

[0061] Figure 4 This is a flowchart of a task scheduling method provided by an embodiment of the present application. The method is applied to a management node in a task scheduling system. The task scheduling system also includes multiple execution nodes. Figure 4 , the method comprises the following steps.

[0062] Step 401: Obtain resource allocation requirements for tasks to be executed.

[0063] In an embodiment of the present application, the management node receives task information of a task to be executed, and the task information includes resource allocation requirements.

[0064] Among them, the resource allocation requirement includes one or more sub-requirements, and the one or more sub-requirements can be expressed in any form. For example, the one or more sub-requirements are expressed in the form of a sub-requirement set, and the sub-requirements in the sub-requirement set can be connected by commas, semicolons, spaces or other symbols. For another example, the one or more sub-requirements are expressed in the form of a logical expression, and the logical expression can be an infix expression or a postfix expression or an expression in other forms. The logical expression method can meet more user needs, and the user can set resource allocation requirements in a more flexible way.

[0065] In an embodiment of the present application, the resource allocation requirement may be represented by a first requirement tag. The first requirement tag includes at least one requirement sub-tag. The at least one requirement sub-tag may be represented in any form. One requirement sub-tag represents one sub-requirement.

[0066] In one implementation, the at least one requirement subtag is in the form of a subtag set. For example, the first requirement tag is 'bigmem, npu, centos', that is, the first requirement tag includes three requirement subtags, and the subtag set formed by these three requirement subtags is the first requirement tag.

[0067] In another implementation, in order to improve the flexibility of users in labeling tasks, reduce the requirements for users to label tasks, and meet more user needs, the first requirement label can be expressed in the form of an infix expression.

[0068] For example, the first requirement tag is 'bigmem', that is, the first requirement tag includes a requirement subtag, and the requirement subtag indicates that the user expects the task to be executed to be executed on an execution node with a large memory. It should be understood that when the first requirement tag includes a requirement subtag, the first requirement tag is also an infix expression. For another example, the first requirement tag is 'bigmem&npu&(centos|ubuntu)', that is, the first requirement tag includes four requirement subtags, and the four requirement subtags are connected by logical operators to form an infix expression, which indicates that the user expects the task to be executed to be executed on an execution node with a large memory and a neural network processing unit (NPU), and the operating system (OS) is centos or ubuntu.

[0069] It can be seen that the infix expression method is the logical expression method of "AND or NOT". Compared with the sub-tag set method to express resource allocation requirements, the logical expression method of "AND or NOT" can meet more user needs and improve the usability and functional completeness of the task scheduling system.

[0070] In an embodiment of the present application, a user may use a command line tool to submit a task, as shown in the following example.

[0071] / / Users submit jobs that need to be scheduled to bigmem, have NPUs, and have an execution node with centos or ubuntu OS

[0072] job_cli–-labels'bigmem&npu&(centos|ubuntu)'myjob.sh

[0073] Of course, users can also submit tasks in other ways. For example, the task scheduling system provides users with a visual operation interface, namely a user interface, which displays multiple types of demand sub-tags for selection. Users can select a demand sub-tag on the user interface, and after selecting a demand sub-tag, submit the task by clicking a submit button on the user interface.

[0074] Exemplarily, multiple types of requirement sub-tags are sequentially displayed on the user interface, and each type of requirement sub-tag has multiple ones. For example, the requirement sub-tags of the operating system include centos, ubuntu, etc. If the user only selects 'centos', it means that the user expects to execute the task on the execution node whose OS is centos. If the user selects 'centos' and 'ubuntu', it means that the user expects to execute the task on the execution node whose OS is centos or ubuntu. In other words, if the user selects at least two requirement sub-tags of the same type, it means that the user expects to execute the task on the execution node that matches any one of the at least two requirement sub-tags.

[0075] In order to reduce the difficulty for users to label tasks, label prompt information can be displayed on the command line interface or the visual operation interface. The label prompt information may include a specific example of the required label, and / or the prompt information of the logical operators of AND, OR, NOT, and brackets. Users can label tasks with reference to the specific example of the required label, or they can choose to use appropriate and accurate logical operators to label tasks according to the prompt information of the logical operators of AND, OR, NOT, and brackets.

[0076] Step 402: confirm whether there is any historical resource allocation requirement in the historical resource allocation information recorded by the management node that is the same as the resource allocation requirement of the task to be executed. The historical resource allocation information is used to record the information of the execution node that matches the resource allocation requirement of the executed task by the management node.

[0077] That is, the management node first searches the recorded historical resource allocation information for the same historical resource allocation requirements as the resource allocation requirements of the task to be executed. The historical resource allocation information is used to record the information of the execution nodes that the management node matches for the resource allocation requirements of the executed tasks. For example, the historical resource allocation information is used to record the correspondence between the requirement label and the matching execution node. The execution nodes that match the requirement representation recorded in the historical resource allocation information can be called candidate execution nodes.

[0078] Since the resource allocation requirements of the tasks to be executed have multiple possible representations, for example, when the resource allocation requirements are represented by requirement tags, requirement tags with the same semantics as a certain requirement tag may have multiple representations. In order to unify the representations for label matching, certain rules can be preset so that these multiple representations can be converted into a unique representation according to the preset rules.

[0079] For example, the first requirement tag of the task to be executed submitted by the user includes at least one requirement subtag, and the at least one requirement subtag may contain English capital letters. In this case, the at least one requirement subtag can be uniformly converted into English lowercase letters according to preset rules. For another example, the first requirement tag includes a subtag set formed by multiple requirement subtags. These multiple requirement subtags may not be arranged in dictionary order. In this case, these multiple requirement subtags can be rearranged in dictionary order according to preset rules. For another example, the first requirement tag is an infix expression including multiple requirement subtags. In this case, the infix expression can be converted into a postfix expression according to preset rules. The postfix expression does not contain English capital letters, and the multiple leaf nodes of each parent node in the tree structure corresponding to the postfix expression are arranged in dictionary order. The above rules can be used alone or in combination according to actual needs. Among them, the postfix expression is also called reverse Polish notation or reverse Polish notation, and reverse Polish notation is more conducive to computer understanding and calculation.

[0080] Figure 5 1 is a schematic diagram of a tree structure represented by reverse Polish notation provided in an embodiment of the present application. The second requirement label represented by the reverse Polish notation is 'bigmem npu¢os ubuntu|&', and the corresponding first requirement label is 'bigmem&npu&(centos|ubuntu)'.

[0081] It should be understood that if the requirement tag submitted by the user is used directly, for example, the requirement tag submitted by the user includes n requirement sub-tags, then the requirement tags with the same semantics as this requirement tag may reach up to n! -1. Exemplarily, the requirement tag submitted by the user is 'ab c', and there are 5 requirement tags with the same semantics as the requirement tag 'ab c', namely 'ac b', 'ba c', 'bc a', 'ca b', and 'cb a'. If the user also uses logical operators such as 'and, or, not, brackets' to represent the requirement tags, then the number of requirement tags with the same semantics as the requirement tags submitted by the user may be greater. If a unified conversion is not performed, the amount of computation for tag matching will increase. Therefore, performing a unified tag conversion according to preset rules can greatly reduce the amount of computation and improve the cache hit rate.

[0082] Based on the above introduction, the management node converts the first requirement tag into the second requirement tag according to the preset rule, and the second requirement tag and the first requirement tag are requirement tags with the same semantics but different representation forms. If the second requirement tag is recorded in the historical resource allocation information, the management node determines that the historical resource allocation information records the historical resource allocation requirements that are the same as the resource allocation requirements of the task to be executed.

[0083] For example, the first requirement tag is 'bigmem&NPU&(ubuntu|centos)', and the preset rule indicates that the first requirement tag is converted into a second requirement tag represented by a suffix expression, all uppercase letters in the first requirement tag are converted into lowercase letters, and multiple leaf nodes of each parent node in the tree structure corresponding to the suffix expression are arranged in dictionary order. Then, the second requirement tag is 'bigmem npu¢os ubuntu|&'.

[0084] In the following, explanation is given by taking the case where the first requirement label is expressed in the form of an infix expression and the second requirement label is expressed in the form of a postfix expression as an example.

[0085] In the embodiment of the present application, the demand tags recorded in the historical resource allocation information are all in the form of suffix expressions, and the corresponding demand tags are recorded according to the tree structure corresponding to the suffix expressions.

[0086] It should be understood that in the application embodiment, the historical resource allocation information records the correspondence between the resource allocation requirements and the candidate execution nodes. In one implementation, the correspondence includes the requirement label and the identifier of the corresponding candidate execution node, wherein the identifier of the execution node can be the node name or other unique identifier of the execution node.

[0087] Exemplarily, a certain execution node is identified as “Node1”.

[0088] In another implementation, the correspondence includes a requirement tag and a corresponding binary string, wherein the binary string includes a multi-bit binary number, and the multi-bit binary number corresponds one-to-one to a plurality of execution nodes in the task scheduling system, and the candidate execution node includes an execution node corresponding to a position where the binary number in the corresponding binary string is a first value. In the embodiment of the present application, the first value is '1'. In some other embodiments, the first value may also be '0'.

[0089] For example, a binary string recorded in the historical resource allocation information is '100010', and these six binary numbers correspond to six execution nodes in the task scheduling system. The candidate execution nodes represented by the binary string include the first execution node and the sixth execution node.

[0090] Optionally, the historical resource allocation information is recorded in the cache of the management node. In this way, the scheduling efficiency can be improved. That is, in the case of a cache hit, the management node directly obtains the target execution node from the cache. Among them, the historical resource allocation information records the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, indicating a cache hit. Of course, the historical resource allocation information can also be recorded in other storage spaces of the management node, or in the storage space of other devices.

[0091] Step 403: When it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, an execution node recorded in the historical resource allocation information and matching the historical resource allocation requirement is determined as a target execution node.

[0092] That is, if the historical resource allocation information records the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, the management node determines the first candidate execution node recorded in the historical resource allocation information as the target execution node. The first candidate execution node refers to the candidate execution node corresponding to the historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed. In the case where the resource allocation requirement of the task to be executed is represented by a first requirement tag, the first candidate execution node refers to the candidate execution node corresponding to the requirement tag (such as the second requirement tag mentioned above) recorded in the historical resource allocation information that has the same semantics as the first requirement tag.

[0093] When it is determined that the second requirement tag is recorded in the historical resource allocation information, the management node determines that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, and determines the first candidate execution node as the target execution node. In this way, there is no need to match a candidate execution node for each task to be executed from multiple execution nodes, which can improve scheduling efficiency.

[0094] Step 404: Schedule the task to be executed to the target execution node for execution.

[0095] Wherein, if the target execution node includes one execution node, the management node directly schedules the task to be executed to the execution node. If the target execution node includes at least two execution nodes, the management node determines at least one execution node from the target execution node according to the specified strategy, and schedules the task to be executed to the at least one execution node. Wherein, the specified strategy is a random selection strategy, that is, the management node randomly selects at least one execution node from the target execution node. Alternatively, the specified strategy is a load balancing strategy, that is, the management node selects at least one execution node from the target execution node according to the load balancing strategy. Alternatively, the specified strategy may also be other strategies, which are not limited in the embodiments of the present application.

[0096] In addition, the number of the at least one execution node may be determined according to the amount of the task to be executed or the node number indication information of the task to be executed acquired by the management node, or may be determined by the management node according to the default setting, or may be determined in other ways. These methods for determining the number of execution nodes may be used alone or in combination.

[0097] It should be understood that if the task to be executed needs to be executed by one execution node, then the management node will select an execution node from the target execution nodes and schedule the task to be executed to the execution node. If the task to be executed needs to be jointly executed by two execution nodes (i.e., cooperative execution), the management node will select two execution nodes from the target execution nodes and schedule the task to be executed to the two execution nodes. And so on.

[0098] In one implementation, the task information of the task to be executed also includes node quantity indication information. For example, the node quantity indication information is a first quantity, and the first quantity refers to the number of execution nodes required to execute the task to be executed. The management node selects a first number of execution nodes from the target execution nodes and schedules the task to be executed to the selected first number of execution nodes for execution.

[0099] In another implementation, the task information of the task to be executed also indicates the task amount. The larger the task amount, the more execution nodes are required. The management node selects an appropriate number of execution nodes from the target execution nodes according to the task amount of the task to be executed.

[0100] If the historical resource allocation information does not record the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, such as the second requirement tag is not recorded, that is, the candidate execution node corresponding to the requirement tag with the same semantics as the first requirement tag is not recorded, then the management node determines the execution node whose node tag matches the resource allocation requirements of the task to be executed from the multiple execution nodes, and determines the execution node whose node tag matches the resource allocation requirements of the task to be executed from the multiple execution nodes as the target execution node. That is, in the case where the historical resource allocation information does not record the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, such as the task whose requirement tag has the same semantics as the first requirement tag has not been received in the history, then the management node needs to re-match the target execution node for the task to be executed from the multiple execution nodes, and cannot directly obtain the target execution node from the historical resource allocation information.

[0101] Among them, if the management node converts the first requirement label into the second requirement label according to the preset rules, if the second requirement label is not recorded in the historical resource allocation information, the management node determines the execution node whose node label matches the second requirement label from the multiple execution nodes, and determines the execution node whose node label matches the second requirement label among the multiple execution nodes as the target execution node.

[0102] It should be understood that, since the first requirement label and the second requirement label have the same semantics, the execution node whose node label matches the first requirement label among the multiple execution nodes is also the execution node that matches the second requirement label.

[0103] As can be seen from the foregoing, the second requirement tag includes at least one requirement sub-tag. There are many ways for the management node to determine the execution node whose node tag matches the second requirement tag from the multiple execution nodes, and one implementation method is described in detail here.

[0104] One implementation method for the management node to determine the execution node whose node label matches the second requirement label from the multiple execution nodes is: obtain the binary strings corresponding to the at least one requirement sub-label respectively from the node bitmap, the node bitmap includes binary strings corresponding to multiple reference labels respectively, each binary string includes a multi-bit binary number corresponding to the multiple execution nodes in the task scheduling system, each binary number indicates whether the corresponding execution node has the corresponding reference label; replace at least one requirement sub-label in the target suffix expression with the corresponding binary string to obtain the expression to be operated, the target suffix expression refers to the suffix expression used to represent the second requirement label; determine the value of the expression to be operated by bit operation to obtain the target binary string; determine the execution node with the corresponding reference label in the target binary string as the execution node whose node label matches the second requirement label among the multiple execution nodes.

[0105] In one example, the task scheduling system includes five execution nodes, and the node labels of the five execution nodes are respectively as shown below.

[0106] Node 0: bigmem, npu, ubuntu

[0107] Node 1: bitmem, centos

[0108] Node 2: npu, ubuntu

[0109] Node 3: bigmem

[0110] Node 4: centos

[0111] In this example, there are four reference labels, namely 'bigmem', 'npu', 'ubuntu' and 'centos'. The node bitmap formed by the above five execution nodes includes four binary strings corresponding to the four reference labels. The four binary strings are '11010', '10100', '01001' and '10100' respectively. The relationship between the reference labels and the node bitmap is shown in Table 1.

[0112] Table 1

[0113]

[0114] Taking the first requirement label as 'bigmem&NPU&(centos|ubuntu)' and the second requirement label as 'bigmemnpu¢os ubuntu|&' as an example, the target suffix expression is 11010 10100&01001 10100|&, and the value of the target suffix expression obtained by bit operation is 10000. Since the target suffix expression has the same semantics as the infix expression representing the first requirement label, in order to facilitate the understanding of the reader, the above operation result is explained here in the calculation method of infix expression according to human thinking, that is, at least one requirement sub-label in the infix expression is replaced with the corresponding binary string to obtain an expression with the same semantics as the above expression to be operated, that is, bigmem&npu&(centos|ubuntu) is replaced with the expression 11010&10100&(01001|10100), and the expression obtained by bit operation is 10000, which is also the value of the above expression to be operated. That is to say, the obtained target binary string is '10000', and only the first bit in the target binary string has a value of '1', indicating that only the first bit has a corresponding reference label. Then the management node determines execution node No. 0 corresponding to the first bit as the execution node whose node label matches the second requirement label among the above five execution nodes, and determines execution node No. 0 as the target execution node.

[0115] In the above example, a position with a value of 1 in the node bitmap indicates that the corresponding execution node has a corresponding reference tag, and a position with a value of 0 indicates that the corresponding execution node does not have a corresponding reference tag.

[0116] It should be understood that the above examples are only used to explain the present solution, and are not used to limit the present solution. In a specific implementation, the reference tags may also include 'arm', 'x86', 'aarch64', etc.

[0117] It can be seen from the above examples that compared with the string comparison method, the embodiment of the present application can greatly reduce the amount of calculation through the tree-structured reverse Polish notation (i.e., suffix expression) and bit operations, thereby improving scheduling efficiency and task execution efficiency.

[0118] Of course, in some other embodiments, such as in some scenarios where efficiency requirements are not particularly high, and in some scenarios where the computing power of the management node is strong, the execution node whose node label matches the second requirement label can be determined from the multiple execution nodes by string comparison.

[0119] In an embodiment of the present application, after determining an execution node among the multiple execution nodes whose node label matches the first requirement label as a target execution node, a correspondence between the first requirement label and the target execution node is added to the historical resource allocation information.

[0120] In an embodiment where the management node converts the first requirement tag into the second requirement tag, the management node adds a correspondence between the second requirement tag and the identifier of the target execution node in the historical resource allocation information, or adds a correspondence between the second requirement tag and the target binary string in the historical resource allocation information.

[0121] By recording a binary string in the historical resource allocation information to represent the corresponding candidate execution node, the amount of stored data can be reduced. When the historical resource allocation information is recorded in the cache of the management node, the waste of the cache can also be reduced.

[0122] It should be understood that the demand tag and binary string recorded in the historical resource allocation information can be regarded as a hash key-value pair, where the demand tag is used as the 'key' and the binary string is used as the 'hash value'. By caching the hash key-value pair, the cache hit rate can be greatly improved and the memory usage can be reduced.

[0123] Optionally, after obtaining the target binary string, the management node directly adds the correspondence between the second requirement tag and the target binary string to the cached historical resource allocation information. Alternatively, after obtaining the target binary string, the management node determines whether the correspondence between the second requirement tag and the target binary string satisfies the cache entry condition. If the correspondence satisfies the cache entry condition, the management node adds the correspondence between the second requirement tag and the target binary string to the cached historical resource allocation information. If the correspondence does not satisfy the cache entry condition, the management node does not add the correspondence between the second requirement tag and the target binary string to the cached historical resource allocation information.

[0124] The cache entry conditions may include the resource allocation requirements recorded in the historical resource allocation information, including the resource allocation requirements whose number of sub-requirements exceeds the first threshold, and / or the resource allocation requirements whose number of requirements exceeds the second threshold, and other conditions may also be included. Thus, when the number of requirement sub-tags included in the second requirement tag does not exceed the first threshold, and / or the number of requirements corresponding to the requirement sub-tags included in the second requirement tag exceeds the second threshold, the management node adds the corresponding relationship between the second requirement tag and the target binary string to the historical resource allocation information.

[0125] Among them, the first threshold can be 3 or 5 or other values. The smaller the first threshold, the smaller the requirement of the corresponding task on the execution node, and users usually do not set many demand sub-tags. In other words, the fewer the demand sub-tags of a task, the closer the task is to the popular demand task, and the cache stores as many candidate execution nodes matching the popular demand tasks as possible, thereby providing a cache hit rate. In other words, the cache hit rate can be improved by limiting the first threshold. Similarly, the second threshold can be a value such as 10 or 15. The larger the second threshold, the higher the corresponding resource requirement, and information with high cache requirements can improve the cache hit rate.

[0126] In addition, since the cache space is limited, the management node may eliminate obsolete cache in the historical resource allocation information according to the least recently used (LRU) principle or other principles, so as to provide cache space for the newly added corresponding relationship.

[0127] In an embodiment of the present application, before using the above-mentioned node bitmap to determine the expression to be operated, the management node first generates a node bitmap based on the node labels of each execution node in the task scheduling system and multiple reference labels. The generated node bitmap represents whether each execution node among the multiple execution nodes has each reference label among the multiple reference labels.

[0128] Before generating the node bitmap, the management node first obtains the node label of each execution node among the multiple execution nodes. There are multiple ways to obtain the node label, such as the administrator manually labeling the execution node and / or the system automatically labeling the execution node.

[0129] In the method where the administrator manually labels the execution nodes, the administrator configures node labels for one or at least two execution nodes through a command line tool or a visual operation interface, wherein one or more node sub-labels may be configured for each execution node, and the configuration information is submitted to the management node. For example, the administrator configures at least two node sub-labels for the first execution node among the multiple execution nodes, generates first configuration information through a command line tool or a visual operation interface, and submits the first configuration information to the management node. For another example, the administrator configures node labels for at least two execution nodes among the multiple execution nodes, and submits second configuration information to the management node.

[0130] The following is an example of how an administrator uses the command line tool to label an execution node.

[0131] / / Add bigmem and npu labels to node1 and node3

[0132] node_cli–-lables'bigmem,npu'node1 node3

[0133] / / Add gpu labels to node1, node2 and node4

[0134] node_cli-l'gpu'node1 node2 node4

[0135] Accordingly, the management node receives first configuration information, the first configuration information includes at least two node sub-labels configured for a first execution node among the multiple execution nodes, the first execution node being any execution node whose node label includes multiple node sub-labels; and / or, the management node receives second configuration information, the second configuration information includes node labels configured for at least two execution nodes among the multiple execution nodes.

[0136] It can be seen that this solution supports configuring multiple node sub-labels for one execution node at a time, and also supports configuring node labels for a batch of execution nodes at a time. In this way, the flexibility of administrators in configuring custom labels can be improved, and the usability and functional completeness of the task scheduling system can be improved.

[0137] In the method where the system automatically labels the execution node, the execution node actively reports its own node information, or sends the node information of the execution node to the management node after receiving the request of the management node, and the management node automatically obtains the information that can be used as the node sub-label from the node information.

[0138] Exemplarily, the node information reported by the first execution node includes 'OS is centos, CPU architecture is x86_64', then the management node can automatically set the node label for the first execution node to include 'centos, x86_64'.

[0139] It should be understood that in order to ensure that the custom labels configured by the administrator do not conflict with the labels automatically added by the system, when the first node sub-label in the configuration information received by the management node conflicts with the node sub-label automatically added by the system, the management node sends a prompt message to the client used by the administrator, indicating that the first node sub-label configured by the administrator conflicts with the node sub-label automatically added by the system.

[0140] Please refer to the following Figure 6 The implementation process of the management node acquiring the node label and generating the node bitmap in the embodiment of the present application is further explained by way of example.

[0141] Figure 6 This is a flow chart of a method for obtaining node labels and generating node bitmaps provided by an embodiment of the present application. Figure 2 As an example, see the resource management interface, resource management module, and job scheduling module shown in Figure 6 The resource management module receives the node label assigned by the administrator through the management interface (i.e., receives the configuration information), and receives the node label reported by the execution node (i.e., the system automatically assigns the label), and constructs the binary string corresponding to the corresponding reference label based on the node label of the execution node, i.e., constructs the bitmap of the corresponding reference label, and constructs the global node bitmap based on the bitmaps of all reference labels. The resource management module synchronizes the node bitmap to the job scheduling module, i.e., the node bitmap is synchronized to the scheduling engine (i.e., the job scheduling module), so that the job scheduling module can match the target execution node for the task to be executed based on the node bitmap.

[0142] Please refer to the following Figure 7 An implementation of the task scheduling method provided in an embodiment of the present application is explained exemplarily.

[0143] Figure 7 is a flowchart of another task scheduling method provided by an embodiment of the present application. Figure 7 The management node receives the task submitted by the user through the job submission interface. The task information of the task includes a requirement tag represented by an infix expression. The management node parses the infix expression through the resource management module, constructs a reverse Polish notation tree structure corresponding to the infix expression, and sends the task to the job scheduling module in the management node, that is, sends the task to the scheduling engine, including sending the constructed tree structure.

[0144] The job scheduling module selects a task from the tasks to be executed for scheduling. The job scheduling module determines to enter the label requirement filtering stage. In this stage, the job scheduling module queries the historical resource allocation information cached by the management node to see whether there are candidate execution nodes corresponding to the demand labels with the same semantics as the demand labels of the tasks to be executed. Simply put, in this stage, the cache is queried first. If the cache hits, the job scheduling module directly obtains the candidate execution nodes corresponding to the demand labels with the same semantics as the demand labels of the tasks to be executed, and obtains the target execution node, that is, directly obtains the candidate execution nodes that meet the label requirements.

[0145] If the cache misses, the job scheduling module determines the target binary string through bit operations based on the node bitmap and the above-mentioned reverse Polish notation, that is, calculates the candidate execution nodes that meet the label requirements. Please refer to the relevant content above for the specific implementation process, which will not be repeated here. After determining the target binary string, the job scheduling module determines the target execution node based on the target binary string. Please refer to the relevant content above for the specific implementation process. At this point, the job scheduling module calculates the node set that meets the demand label requirements. The execution nodes in this node set are the target execution nodes, and the target execution nodes may include one execution node or at least two execution nodes.

[0146] After determining the target execution node, the job scheduling module performs subsequent scheduling processes based on the target execution node, for example, determines at least one execution node from the target execution nodes according to a specified strategy, and schedules the task to be executed to the at least one execution node.

[0147] In addition, the job scheduling module also determines whether the correspondence between the above-mentioned reverse Polish notation and the target binary string meets the cache entry condition. If the correspondence meets the cache entry condition, the job scheduling module adds the correspondence between the above-mentioned reverse Polish notation and the target binary string to the historical resource allocation information, that is, enters the cache. In addition, since the cache space is limited, the job scheduling module can eliminate the obsolete cache in the historical resource allocation information according to the LRU principle, so as to provide cache space for the newly added correspondence.

[0148] In summary, in the embodiment of the present application, after obtaining the resource allocation requirements of the task to be executed, if the historical resource allocation information records the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, the execution node that matches the historical resource allocation requirements recorded in the historical resource allocation information is directly determined as the target execution node. In this way, there is no need to re-determine the execution node whose node label matches the resource allocation requirements of the task to be executed from multiple execution nodes, and there is no need for a large amount of string matching calculations, which improves the screening efficiency of the execution nodes, thereby improving the scheduling efficiency and the execution efficiency of the task.

[0149] In the specific implementation, users can use AND or NOT logical syntax to describe the resource allocation requirements of tasks, and support administrators to customize labels for execution nodes, label multiple node sub-labels for one execution node at a time, and label multiple execution nodes in sequence (i.e., set operations). The management node uses bit operations and reverse Polish notation to convert label scheduling into efficient bit operations to improve performance. Hash key-value pairs are constructed to cache scheduling results, and reverse Polish notation is used to merge label requirements with the same semantics to improve cache hit rate.

[0150] Figure 8 is a structural diagram of a task scheduling device 800 provided in an embodiment of the present application. The task scheduling device 800 can be implemented by software, hardware, or a combination of both to form part or all of a computer device. The computer device can be Figure 1 or Figure 2 In the embodiment of the present application, the device 800 is included in the management node of the task scheduling system, and the task scheduling system also includes multiple execution nodes. Figure 8 The device 800 includes: a first acquisition module 801, a first determination module 802, a second determination module 803 and a scheduling module 804.

[0151] The first acquisition module 801 is used to acquire the resource allocation requirements of the task to be executed;

[0152] A first determination module 802 is used to confirm whether there is a historical resource allocation requirement in the historical resource allocation information recorded by the management node that is the same as the resource allocation requirement of the task to be executed, and the historical resource allocation information is used to record the information of the execution node that the management node is matched with the resource allocation requirement of the executed task;

[0153] The second determination module 803 is used to determine the execution node matching the historical resource allocation requirement recorded in the historical resource allocation information as the target execution node when it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed;

[0154] The scheduling module 804 is used to schedule the tasks to be executed to the target execution node for execution.

[0155] Optionally, the resource allocation requirement includes multiple sub-requirements, and the multiple sub-requirements are expressed in the form of logical expressions.

[0156] Optionally, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold.

[0157] Optionally, the historical resource allocation demands recorded in the historical resource allocation information include resource allocation demands whose demand times exceed a second threshold.

[0158] Optionally, the resource allocation requirement of the task to be executed is represented by a first requirement tag; the apparatus 800 further includes:

[0159] A label conversion module, used to convert the first requirement label into a second requirement label according to a preset rule, wherein the second requirement label and the first requirement label have the same semantics but different representation forms;

[0160] The first determination module is specifically configured to determine that the historical resource allocation information records a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed if the historical resource allocation information records a second requirement tag.

[0161] Optionally, the device 800 further includes:

[0162] A demand matching module, configured to determine an execution node whose node label matches the second demand label from a plurality of execution nodes if the second demand label is not recorded in the historical resource allocation information;

[0163] The third determination module is used to determine an execution node whose node label matches the second requirement label among the multiple execution nodes as a target execution node.

[0164] Optionally, the first requirement tag is expressed in the form of an infix expression, and the second requirement tag is expressed in the form of a postfix expression.

[0165] Optionally, the second requirement tag includes at least one requirement sub-tag;

[0166] The demand matching module is specifically used for:

[0167] Obtaining binary strings corresponding to the at least one requirement sub-tag from a node bitmap, wherein the node bitmap includes binary strings corresponding to multiple reference tags, each binary string includes multiple binary digits corresponding to the multiple execution nodes one by one, and each binary digit indicates whether the corresponding execution node has a corresponding reference tag;

[0168] The at least one requirement sub-tag in the target suffix expression is replaced with a corresponding binary string to obtain an expression to be operated, wherein the target suffix expression refers to a suffix expression used to represent the second requirement tag;

[0169] Determine the value of the expression to be operated through bit operation to obtain the target binary string;

[0170] An execution node having a corresponding reference label in the target binary string is determined as an execution node whose node label matches the second requirement label among the multiple execution nodes.

[0171] Optionally, the device 800 further includes:

[0172] The adding module is used to add the corresponding relationship between the second requirement label and the target binary string in the historical resource allocation information.

[0173] Optionally, the device 800 further includes:

[0174] A second acquisition module is used to acquire a node label of each execution node among the multiple execution nodes;

[0175] A generation module is used to generate a node bitmap based on the node label of each execution node in the multiple execution nodes and the multiple reference labels, and the node bitmap represents whether each execution node in the multiple execution nodes has each reference label in the multiple reference labels.

[0176] Optionally, the second acquisition module is specifically used for:

[0177] receiving first configuration information, the first configuration information including at least two node sub-tags configured for a first execution node among the multiple execution nodes, the first execution node being any execution node whose node tag includes the multiple node sub-tags; and / or,

[0178] Second configuration information is received, where the second configuration information includes node labels configured for at least two execution nodes among the plurality of execution nodes.

[0179] Optionally, historical resource allocation information is recorded in a cache of the management node.

[0180] In an embodiment of the present application, after obtaining the resource allocation requirements of the task to be executed, if the historical resource allocation information records the same historical resource allocation requirements as the resource allocation requirements of the task to be executed, the execution node recorded in the historical resource allocation information that matches the historical resource allocation requirements is directly determined as the target execution node. In this way, there is no need to re-determine the execution node whose node label matches the resource allocation requirements of the task to be executed from multiple execution nodes, and there is no need for a large amount of string matching calculations, which improves the screening efficiency of the execution nodes, thereby improving the scheduling efficiency and the execution efficiency of the task.

[0181] It should be noted that: the task scheduling device provided in the above embodiment only uses the division of the above functional modules as an example when performing task scheduling. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the task scheduling device provided in the above embodiment and the task scheduling method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0182] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0183] It should be understood that the "at least one" mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not limit them to be necessarily different.

[0184] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0185] The above-mentioned embodiments are provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A task scheduling method, characterized in that: The method is applied to a management node in a task scheduling system, wherein the task scheduling system further includes a plurality of execution nodes. Obtain resource allocation requirements for tasks to be performed; Confirm whether there is a historical resource allocation requirement in the historical resource allocation information recorded by the management node that is the same as the resource allocation requirement of the task to be executed, the historical resource allocation information is used to record the information of the execution node that the management node matches the resource allocation requirement of the executed task; When it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, determining an execution node that matches the historical resource allocation requirement and is recorded in the historical resource allocation information as a target execution node; The task to be executed is dispatched to the target execution node for execution.

2. The method according to claim 1, characterized in that The resource allocation requirement includes multiple sub-requirements, and the multiple sub-requirements are expressed in the form of logical expressions.

3. The method according to claim 1 or 2, characterized in that The historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold.

4. The method according to any one of claims 1 to 3, characterized in that: The historical resource allocation demands recorded in the historical resource allocation information include resource allocation demands whose demand times exceed a second threshold.

5. The method according to any one of claims 1 to 4, characterized in that: The resource allocation requirement of the task to be executed is represented by a first requirement tag; the method further comprises: Converting the first requirement tag into a second requirement tag according to a preset rule, wherein the second requirement tag and the first requirement tag have the same semantics but different representation forms; If the second requirement tag is recorded in the historical resource allocation information, it is determined that the historical resource allocation information records a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed.

6. The method according to claim 5, characterized in that The method further comprises: If the second requirement tag is not recorded in the historical resource allocation information, determining an execution node whose node tag matches the second requirement tag from the multiple execution nodes; An execution node among the multiple execution nodes whose node label matches the second requirement label is determined as the target execution node.

7. The method according to claim 5 or 6, characterized in that The first requirement tag is expressed in the form of an infix expression, and the second requirement tag is expressed in the form of a postfix expression.

8. The method according to claim 6, characterized in that The second requirement tag includes at least one requirement sub-tag; The determining, from the multiple execution nodes, an execution node whose node label matches the second requirement label includes: Obtaining binary strings corresponding to the at least one requirement sub-tag from a node bitmap, wherein the node bitmap includes binary strings corresponding to multiple reference tags, each binary string includes multiple binary digits corresponding to the multiple execution nodes one by one, and each binary digit indicates whether the corresponding execution node has a corresponding reference tag; Replacing the at least one requirement sub-tag in the target suffix expression with a corresponding binary string to obtain an expression to be operated, wherein the target suffix expression refers to a suffix expression used to represent the second requirement tag; Determine the value of the expression to be operated by bit operation to obtain a target binary string; An execution node having a corresponding reference label in the target binary string is determined as an execution node whose node label matches the second requirement label among the multiple execution nodes.

9. The method according to claim 8, characterized in that The method further comprises: A correspondence between the second requirement tag and the target binary character string is added to the historical resource allocation information.

10. The method according to claim 8 or 9, characterized in that The method further comprises: Obtaining a node label of each execution node among the multiple execution nodes; The node bitmap is generated based on the node label of each execution node among the multiple execution nodes and the multiple reference labels, and the node bitmap indicates whether each execution node among the multiple execution nodes has each reference label among the multiple reference labels.

11. The method according to claim 10, characterized in that The obtaining of the node label of each execution node among the plurality of execution nodes comprises: receiving first configuration information, wherein the first configuration information includes at least two node sub-labels configured for a first execution node among the plurality of execution nodes, the first execution node being any execution node whose node label includes a plurality of node sub-labels; and / or, Second configuration information is received, where the second configuration information includes node labels configured for at least two execution nodes among the plurality of execution nodes.

12. The method according to any one of claims 1 to 11, characterized in that: The historical resource allocation information is recorded in the cache of the management node.

13. A task scheduling device, characterized in that: A management node included in a task scheduling system, the task scheduling system also includes a plurality of execution nodes, the device includes: A first acquisition module is used to acquire resource allocation requirements of tasks to be executed; A first determination module is used to confirm whether there is a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed in the historical resource allocation information recorded by the management node, and the historical resource allocation information is used to record the information of the execution node that the management node matches the resource allocation requirement of the executed task; A second determination module is used to determine, when it is determined that there is a historical resource allocation requirement in the historical resource allocation information that is the same as the resource allocation requirement of the task to be executed, an execution node recorded in the historical resource allocation information and matching the historical resource allocation requirement as a target execution node; The scheduling module is used to schedule the task to be executed to the target execution node.

14. The device according to claim 13, characterized in that The historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold.

15. The device according to claim 13 or 14, characterized in that The resource allocation requirement of the task to be executed is represented by a first requirement tag; the device also includes: A label conversion module, used to convert the first requirement label into a second requirement label according to a preset rule, wherein the second requirement label and the first requirement label have the same semantics but different representation forms; The first determining module is specifically configured to determine that a historical resource allocation requirement that is the same as the resource allocation requirement of the task to be executed is recorded in the historical resource allocation information if the second requirement tag is recorded in the historical resource allocation information.

16. The device according to claim 15, characterized in that The device also includes: a demand matching module, configured to determine, from the plurality of execution nodes, an execution node whose node label matches the second demand label if the second demand label is not recorded in the historical resource allocation information; The third determination module is used to determine an execution node whose node label matches the second requirement label among the multiple execution nodes as the target execution node.

17. The device according to claim 15 or 16, characterized in that The first requirement tag is expressed in the form of an infix expression, and the second requirement tag is expressed in the form of a postfix expression.

18. A task scheduling system, characterized in that: The task scheduling system includes a management node and multiple execution nodes; The management node is used to execute the steps of the method according to any one of claims 1 to 12.

19. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

20. A computer program product, characterized in that The computer program product stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images