Task scheduling method, device, system, storage medium and computer program product

Abstract

The application discloses a task scheduling method, device and system, a storage medium and a computer program product, and belongs to the technical field of computers. The method comprises the following steps: after obtaining resource allocation requirements of a to-be-executed task, if historical resource allocation information records historical resource allocation requirements which are the same as the resource allocation requirements of the to-be-executed task, an execution node which matches the historical resource allocation requirements and is recorded in the historical resource allocation information is directly determined as a target execution node. In this way, it is not necessary to determine an execution node whose node label matches the resource allocation requirements of the to-be-executed task from multiple execution nodes again, a large amount of string matching calculation is not necessary, the screening efficiency of the execution node is improved, and therefore the scheduling efficiency and the execution efficiency of the task are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a task scheduling method, device and system, a storage medium and a computer program product. BACKGROUND

[0002] Currently, a user can encapsulate a business logic into a task through a client, and submit related information of the task to a management node in a task scheduling system. The task scheduling system further includes a plurality of execution nodes, and the management node is responsible for scheduling each task submitted by the user to at least one execution node in the plurality of execution nodes according to a scheduling strategy, so that the at least one execution node executes the task. A reasonable and efficient scheduling method can improve the resource utilization rate of the task scheduling system and improve the execution efficiency of the task.

[0003] With the research and development of scheduling technology, a scheduling technology based on node labels emerges as the times require. In the related technology of task scheduling based on node labels, each execution node has a node label, and the node label represents one or more information of a central processing unit (CPU) architecture, an operating system type, whether to include a heterogeneous processor, whether to include a graphics processing unit (GPU), and the like. The user can specify a requirement label when encapsulating the task, so that the related information of the task submitted by the user includes the requirement label. After receiving the task with the requirement label, the management node matches the requirement label with the node label of each execution node in the plurality of execution nodes through string comparison, so as to filter out the execution node matched with the task, and then schedules the task to the filtered execution node. For example, the requirement label of the task submitted by the user includes 'x86', which indicates that the user hopes to execute the task on an execution node with an x86 architecture. Then, the management node filters out all execution nodes with an x86 architecture in the task scheduling system through string comparison, and then filters out the execution node for executing the task from the execution nodes with an x86 architecture.

[0004] However, the number of tasks to be processed by the task scheduling system is huge, and the number of execution nodes in the task scheduling system is also relatively large. The above method needs to compare the requirement label of each task with the node label of each execution node, which leads to a relatively low filtering efficiency and affects the scheduling efficiency. Moreover, the string comparison method for filtering the execution node has a large amount of calculation, which also affects the scheduling efficiency and thus affects the execution efficiency of the task. SUMMARY

[0005] The application provides a task scheduling method and device, a storage medium and a computer program product, which can improve scheduling efficiency and task execution efficiency. The technical solution is as follows:

[0006] In a first aspect, a task scheduling method is provided, which is applied to a management node in a task scheduling system, the task scheduling system further comprising a plurality of execution nodes, and the method comprises the following steps:

[0007] obtaining resource allocation requirements of a to-be-executed task; determining whether there is a historical resource allocation requirement same as the resource allocation requirements of the to-be-executed task in historical resource allocation information recorded by the management node, the historical resource allocation information being used to record information of an execution node matched by the management node for resource allocation requirements of an executed task; when it is determined that there is a historical resource allocation requirement same as the resource allocation requirements of the to-be-executed task in the historical resource allocation information, determining an execution node recorded in the historical resource allocation information and matched with the historical resource allocation requirement as a target execution node; and scheduling the to-be-executed task to the target execution node for execution.

[0008] That is, if there is a historical resource allocation requirement same as the resource allocation requirements of the to-be-executed task recorded in the historical resource allocation information, the execution node recorded in the historical resource allocation information and matched with the historical resource allocation requirement is directly determined as the target execution node, so that it is not necessary to determine the execution node with a node tag matched with the resource allocation requirements of the to-be-executed task from the plurality of execution nodes again, and a large amount of string matching calculation is not necessary, which improves the screening efficiency of the execution node and thus improves the scheduling efficiency and the execution efficiency of the task.

[0009] Optionally, the resource allocation requirements comprise a plurality of sub-requirements, and the plurality of sub-requirements are expressed in a logical expression manner. Compared with the sub-tag set manner for expressing the resource allocation requirements, the logical expression manner for expressing the resource allocation requirements can meet more user requirements and improve the ease of use and the functional completeness of the task scheduling system.

[0010] Optionally, the historical resource allocation requirements recorded in the historical resource allocation information comprise resource allocation requirements with a number of sub-requirements exceeding a first threshold. And / or, the historical resource allocation requirements recorded in the historical resource allocation information comprise resource allocation requirements with a number of requirements exceeding a second threshold. In this way, the hit rate of obtaining the target execution node from the historical resource allocation information can be improved.

[0011] Optionally, the resource allocation requirement of the task to be executed is represented by a first requirement tag, and the first requirement tag is represented in a way of infix expression. The way of infix expression is a logical expression way of 'and or not'. Compared with representing the requirement tag in a way of sub-tag set, representing the requirement tag in the way of 'and or not' logic expression can satisfy more user requirements and improve the ease of use and the functional completeness of the task scheduling system.

[0012] Since a requirement tag with the same semantic as a certain requirement tag can have multiple representations, in order to unify the representation and facilitate tag matching, a certain rule can be preset so that the multiple representations can be converted into a unique representation according to the preset rule.

[0013] Optionally, the method further includes: converting the first requirement tag into a second requirement tag according to the preset rule, the second requirement tag being a requirement tag with the same semantic as the first requirement tag but different representation; and determining that the historical resource allocation information records a historical resource allocation requirement same as the resource allocation requirement of the task to be executed if the second requirement tag is recorded in the historical resource allocation information. The unified tag conversion according to the preset rule can greatly reduce the computational complexity and improve the cache hit rate.

[0014] In the case where the first requirement tag is represented in a way of infix expression, the second requirement tag is represented in a way of postfix expression. The postfix expression is also called reverse polish or reverse polish expression, and the reverse polish is more conducive to the understanding and calculation of computers.

[0015] Optionally, the method further includes: determining an execution node whose node tag matches the second requirement tag from the plurality of execution nodes if the second requirement tag is not recorded in the historical resource allocation information; and determining the execution node whose node tag matches the second requirement tag as the target execution node.

[0016] The second demand label includes at least one demand sub-label; determining an execution node in the plurality of execution nodes, in which the node label of the execution node matches the second demand label, includes: obtaining a binary string corresponding to each demand sub-label from a node bitmap, the node bitmap includes a binary string corresponding to each reference label, each binary string includes a plurality of binary numbers corresponding to the plurality of execution nodes, and each binary number indicates whether the corresponding execution node has the corresponding reference label; replacing at least one demand sub-label in the target postfix expression with the corresponding binary string to obtain a to-be-operated expression, the target postfix expression being a postfix expression used to represent the second demand label; determining the value of the to-be-operated expression through bit operation to obtain a target binary string; and determining the execution node having the corresponding reference label in the target binary string as the execution node in the plurality of execution nodes, in which the node label of the execution node matches the second demand label. In this way, the label scheduling can be converted into efficient bit operation by using bit operation and Polish notation, thereby improving performance.

[0017] Optionally, a corresponding relationship between the second demand label and the target binary string is added in the historical resource allocation information. In this way, the amount of data stored can be reduced by recording the binary string in the historical resource allocation information to represent the corresponding execution node.

[0018] Optionally, the historical resource allocation information is recorded in the cache of the management node. In this way, the scheduling efficiency can be improved by using the cache.

[0019] Optionally, the method further includes: obtaining the node label of each execution node in the plurality of execution nodes; and generating a node bitmap based on the node label of each execution node in the plurality of execution nodes and the plurality of reference labels, the node bitmap representing whether each execution node in the plurality of execution nodes has each reference label in the plurality of reference labels.

[0020] The node label of each execution node in the plurality of execution nodes is obtained by: receiving first configuration information, the first configuration information including at least two node sub-labels configured for a first execution node in the plurality of execution nodes, the first execution node being any execution node in which the node label includes the plurality of node sub-labels; and / or receiving second configuration information, the second configuration information including the node label configured for at least two execution nodes in the plurality of execution nodes. This solution supports configuring multiple node sub-labels for one execution node at a time, and also supports configuring the node label for a batch of execution nodes at a time. In this way, the flexibility of the administrator in configuring the custom label can be improved, and the ease of use and the functional completeness of the task scheduling system can be improved.

[0021] Optionally, the task to be executed is scheduled to be executed by a target execution node, including: determining at least one first target execution node from the target execution nodes, and scheduling the task to be executed by the at least one first target execution node. That is, the target execution nodes are candidate execution nodes, and the management node further selects at least one execution node from the candidate execution nodes for executing the task. The further selection strategy can include a load balancing strategy or a random selection strategy, etc. The number of the at least one execution node can be determined according to the task amount of the task to be executed or the node number indication information of the task to be executed, or can also be determined by the management node according to a default setting. These several ways of determining the number of execution nodes can be used alone or in combination.

[0022] In a second aspect, a task scheduling apparatus is provided, which has functions to implement the behaviors of the task scheduling method in the first aspect. The task scheduling apparatus includes one or more modules for implementing the task scheduling method provided in the first aspect.

[0023] In a third aspect, a task scheduling system is provided, which includes a management node and a plurality of execution nodes. The management node is configured to perform the steps of the task scheduling method provided in the first aspect. The plurality of execution nodes are configured to execute the tasks allocated by the management node.

[0024] In a fourth aspect, a computer device is provided, which includes a processor and a memory. The memory is configured to store a program for executing the task scheduling method provided in the first aspect, and store data related to the task scheduling method provided in the first aspect. The processor is configured to execute the program stored in the memory. The computer device can further include a communication bus for establishing a connection between the processor and the memory.

[0025] In a fifth aspect, a computer readable storage medium is provided, which stores instructions, and when the instructions are run on a computer, the computer executes the task scheduling method provided in the first aspect.

[0026] In a sixth aspect, a computer program product is provided, which includes instructions, and when the instructions are run on a computer, the computer executes the task scheduling method provided in the first aspect.

[0027] The technical effects obtained by the second aspect to the sixth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1is a system architecture diagram related to a task scheduling method provided by an embodiment of the present application;

[0029] Figure 2 is a system architecture diagram related to another task scheduling method provided by an embodiment of the present application;

[0030] Figure 3 is a structural schematic diagram of a computer device provided by an embodiment of the present application;

[0031] Figure 4 is a flowchart of a task scheduling method provided by an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of a reverse polish notation represented by a tree structure provided in an embodiment of the present application;

[0033] Figure 6 is a flowchart of a method for obtaining a node label and generating a node bitmap provided by an embodiment of the present application;

[0034] Figure 7 is a flowchart of another task scheduling method provided by an embodiment of the present application;

[0035] Figure 8 is a structural schematic diagram of a task scheduling device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0037] Figure 1 is a system architecture diagram related to a task scheduling method provided by an embodiment of the present application. The system can be referred to as a task scheduling system, or simply a scheduling system. Referring to Figure 1 , the system includes a management node 101 and a plurality of execution nodes 102. The management node 101 and the plurality of execution nodes 102 are both established with wireless or wired communication connection.

[0038] The management node 101 is configured to obtain task information of a to-be-executed task, the task information including resource allocation requirements, determine a target execution node 102 for the to-be-executed task by using the present solution, and schedule the to-be-executed task to the target execution node 102 for execution. Optionally, the management node 101 is configured to schedule the to-be-executed task to at least one execution node 102 in the target execution node 102 for execution.

[0039] Each execution node 102 in the plurality of execution nodes 102 is configured to execute a task allocated by the management node 101.

[0040] In the embodiments of the present application, the plurality of execution nodes 102 can be a plurality of computing nodes, a plurality of network nodes, a plurality of computing power network elements, a plurality of grids, or a plurality of computing processes. The network nodes can include switches, routers, and other network devices. The computing power network elements and the grids can be upper-level concepts of the computing nodes. For example, a computing power network element can include a plurality of computing nodes. The computing processes can be Linux processes. That is, the present scheme can be applied to various scheduling scenarios, including various distributed cluster computing scenarios, such as high-performance computing (HPC) clusters, artificial intelligence (AI), big data, cloud services, micro-service scheduling, network switching, and the like.

[0041] Next, please refer to Figure 2 The modules included in the management node and the modules included in the execution node are introduced.

[0042] Figure 2 is another system architecture diagram involved in a task scheduling method provided in the embodiments of the present application. The system includes a management node and a plurality of execution nodes, Figure 2 Any one of the plurality of execution nodes is shown in

[0043] Referring to Figure 2 , the management node includes an application processing interface (API) interaction module, a resource management module, and a job scheduling module. In some scenarios, the job can also be referred to as a task. The API interaction module includes a resource management interface and a job submission interface. The job scheduling module can also be referred to as a scheduling engine, a task scheduling module, and the like.

[0044] The resource management interface is used by an administrator. The administrator can configure node tags for the execution nodes through the resource management interface, that is, the resource management interface is used to receive configuration information, and the configuration information includes the node tags configured by the administrator for the execution nodes. Alternatively, the administrator can also view, delete, and modify the node tags of the execution nodes through the resource management interface, that is, the resource management interface is also used to receive a node tag query request and / or a node tag update request, and return corresponding information.

[0045] The job submission interface is used by a user. The user can submit a task through the job submission interface. The task information of the submitted task can include resource allocation requirements, which can be represented by requirement tags, that is, the job submission interface is used to receive the task information of the task, and the task information includes the requirement tags.

[0046] The resource management module is configured to obtain node labels of each of the plurality of execution nodes, including obtaining configuration information submitted by an administrator, and optionally, further including node information of the execution nodes automatically reported by the execution nodes or automatically collected by the resource management module, and obtaining a part of node sub-labels included in the node labels from the node information. The resource management module is further configured to generate a node bitmap based on the node labels of each of the plurality of execution nodes and a plurality of reference labels, and synchronize the node bitmap to the job scheduling module. The node bitmap indicates whether each of the plurality of execution nodes has each of the plurality of reference labels. For specific implementation manners, refer to the detailed description of the method embodiments below.

[0047] The job scheduling module is configured to schedule tasks based on the node bitmap using the present solution, including determining target execution nodes for a to-be-executed task, and scheduling the to-be-executed task to at least one of the target execution nodes, i.e., issuing the task to the execution nodes.

[0048] The execution node includes a job control module and an operating system. The job control module is configured to receive tasks issued by the management node, specifically, receive tasks issued by the job scheduling module, and run the received tasks to the operating system, i.e., execute the corresponding tasks through the operating system.

[0049] Optionally, the execution node further includes a resource monitoring module. The resource monitoring module is configured to collect node information of the execution node from the operating system, including but not limited to the following information: node name, performance information, operating system information, processor information, etc., wherein the operating system information and the processor information can be used as node sub-labels. The resource monitoring module is further configured to actively report the node information of the execution node to the management node, or send the node information of the execution node to the management node after receiving a request from the management node. As an example, the execution node sends the node information to the management node when registering with the task scheduling system and / or when going online.

[0050] Optionally, the resource monitoring module collects node information from the operating system as original information, and the resource monitoring system reports the original information to the management node after processing the original information, for example, formats the original information and encapsulates it into a message format.

[0051] The embodiments of the present application also provide a scheduler, as shown in Figure 2 , the scheduler includes a management side and a computing side. The management side includes the resource management interface, the job submission interface, the resource management module and the job scheduling module introduced above. The computing side includes the resource monitoring module and the job control module introduced above. The scheduler can be regarded as a brain of a distributed computing cluster, mainly responsible for allocating corresponding system resources for tasks submitted by users. By Figure 2It can be seen that the scheduler provides rich functions and interfaces, which facilitates users and administrators to use and maintain, and can improve system resource utilization, scheduling efficiency and task execution efficiency.

[0052] It should be understood that the system architecture and business scenarios described in the embodiments of the application are for more clearly illustrating the technical solutions of the embodiments of the application, and do not constitute a limitation on the technical solutions provided by the embodiments of the application. Those skilled in the art can know that, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the application are also applicable to similar technical problems.

[0053] Please refer to Figure 3 , Figure 3 is a structural schematic diagram of a computer device according to an embodiment of the application. Optionally, the computer device is a management node or an execution node shown in Figure 1 , and the computer device includes one or more processors 301, a communication bus 302, a memory 303, and one or more communication interfaces 304.

[0054] The processor 301 is a general central processing unit (CPU), a network processing (NP), a microprocessor, or one or more integrated circuits for implementing the solutions of the application, such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. Optionally, the PLD is a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0055] The communication bus 302 is used to transmit information between the above-mentioned components. Optionally, the communication bus 302 is divided into an address bus, a data bus, a control bus, and the like. For the convenience of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus.

[0056] Optionally, the memory 303 is a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), a digital versatile disc, a Blu-ray disc, and the like, a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory 303 exists independently, and is connected with the processor 301 through the communication bus 302, or the memory 303 is integrated with the processor 301.

[0057] The communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks. The communication interface 304 includes a wired communication interface, and optionally also includes a wireless communication interface. The wired communication interface is, for example, an Ethernet interface, and the like. Optionally, the Ethernet interface is an optical interface, an electrical interface, or a combination thereof. The wireless communication interface is, for example, a wireless local area networks (WLAN) interface, a cellular network communication interface, or a combination thereof, and the like.

[0058] Optionally, in some embodiments, the computer device includes multiple processors, such as the processor 301 and the processor 305 as shown in FIG. 3. Each of these processors is a single-core processor, or a multi-core processor. Optionally, the processor herein refers to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Figure 3

[0059] In a specific implementation, as an embodiment, the computer device further includes an output device 306 and an input device 307. The output device 306 communicates with the processor 301, and can display information in various ways. For example, the output device 306 is a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, a projector, and the like. The input device 307 communicates with the processor 301, and can receive user input in various ways. For example, the input device 307 is a mouse, a keyboard, a touch screen device, a sensor device, and the like.

[0060] ​In some embodiments, the memory 303 is configured to store program code 310 for implementing the solutions of the present application, and the processor 301 is configured to execute the program code 310 stored in the memory 303. The program code includes one or more software modules, and the computer device is configured to implement the following steps by means of the processor 301 and the program code 310 in the memory 303. Figure 4 The embodiments provide a task scheduling method.

[0061] Figure 4 is a flowchart of a task scheduling method provided by an embodiment of the present application. The method is applied to a management node in a task scheduling system, and the task scheduling system further includes a plurality of execution nodes. Please refer to Figure 4 The method includes the following steps.

[0062] Step 401: Obtain resource allocation requirements of a task to be executed.

[0063] In an embodiment of the present application, the management node receives task information of a task to be executed, and the task information includes resource allocation requirements.

[0064] The resource allocation requirements include one or more sub-requirements, which can be expressed in any form. For example, the one or more sub-requirements are expressed in the form of a sub-requirement set, and the sub-requirements in the sub-requirement set can be connected by a comma, a colon, a space or other symbols. For another example, the one or more sub-requirements are expressed in the form of a logical expression, which can be an infix expression or a postfix expression or other forms of expression. The form of a logical expression can meet more user requirements, and the user can set the resource allocation requirements in a more flexible manner.

[0065] In an embodiment of the present application, the resource allocation requirements can be expressed by a first requirement tag. The first requirement tag includes at least one requirement sub-tag. The at least one requirement sub-tag can be expressed in any form. One requirement sub-tag represents one sub-requirement.

[0066] In one implementation, the at least one requirement sub-tag is in the form of a sub-tag set. For example, the first requirement tag is ‘bigmem, npu, centos’, that is, the first requirement tag includes three requirement sub-tags, and the sub-tag set formed by the three requirement sub-tags is the first requirement tag.

[0067] In another implementation, in order to improve the flexibility of the user in tagging the task, reduce the requirements of the user in tagging the task, and meet more user requirements, the first requirement tag can be expressed in the form of an infix expression.

[0068] For example, the first requirement label is 'bigmem', that is, the first requirement label includes one requirement sub-label, which indicates that the user expects the to-be-executed task to be executed on an execution node with large memory. It should be understood that in the case where the first requirement label includes one requirement sub-label, the first requirement label is also a infix expression. For another example, the first requirement label is 'bigmem&npu&(centos|ubuntu)', that is, the first requirement label includes four requirement sub-labels, which are connected by logical operators to form an infix expression, which indicates that the user expects the to-be-executed task to be executed on an execution node with large memory and a neural network processing unit (NPU), and the operating system (OS) is centos or ubuntu.

[0069] It can be seen that the infix expression is a way of 'and or not' logical expression. Compared with the way of sub-label set to represent the resource allocation requirement, the way of 'and or not' logical expression to represent the resource allocation requirement can meet more user requirements, and improve the ease of use and functional completeness of the task scheduling system.

[0070] In the embodiments of the present application, the user can use a command line tool to submit a task, for example as follows.

[0071] / / The user submits a job, which needs to be scheduled to an execution node with bigmem, NPU, and the OS is centos or ubuntu

[0072] job_cli–-labels 'bigmem&npu&(centos|ubuntu)' myjob.sh

[0073] Of course, the user can also submit a task in other ways. For example, the task scheduling system provides a visual operation interface, that is, a user interface, for the user, the user interface displays a plurality of types of requirement sub-labels that can be selected, the user can select the requirement sub-label on the user interface, and after selecting the requirement sub-label, the user submits the task by clicking a submit button on the user interface.

[0074] For example, a plurality of types of requirement sub-labels are displayed in sequence on the user interface, and each type of requirement sub-label has a plurality of sub-labels. For example, the requirement sub-labels of the operating system include centos, ubuntu, etc. If the user selects only 'centos', it indicates that the user expects to execute the task on an execution node with the OS of centos. If the user selects 'centos' and 'ubuntu', it indicates that the user expects to execute the task on an execution node with the OS of centos or ubuntu. That is, if the user selects at least two requirement sub-labels of the same type, it indicates that the user expects to execute the task on an execution node matching any one of the at least two requirement sub-labels.

[0075] In order to reduce the difficulty of the user in tagging the task, the command line interface or the visual operation interface can display a label prompt information. The label prompt information can include a specific requirement label example and / or a prompt information of the logical operators of AND, OR, NOT and parentheses. The user can refer to the specific requirement label example to tag the task, or can select a suitable and accurate logical operator according to the prompt information of the logical operators of AND, OR, NOT and parentheses to tag the task.

[0076] Step 402: confirming whether there is a historical resource allocation requirement same as the resource allocation requirement of the to-be-executed task in the historical resource allocation information recorded by the management node. The historical resource allocation information is used to record the information of the execution node matched by the management node for the resource allocation requirement of the executed task.

[0077] That is, the management node first searches the recorded historical resource allocation information for a historical resource allocation requirement same as the resource allocation requirement of the to-be-executed task. The historical resource allocation information is used to record the information of the execution node matched by the management node for the resource allocation requirement of the executed task. For example, the historical resource allocation information is used to record the correspondence between the requirement label and the matched execution node. The execution node recorded in the historical resource allocation information and matched with the requirement label can be referred to as a candidate execution node.

[0078] Since the resource allocation requirement of the to-be-executed task has a plurality of possible forms of representation, for example, in the case where the resource allocation requirement is represented by a requirement label, there can be a plurality of forms of representation of the requirement label with the same semantics. In order to unify the form of representation and facilitate label matching, a certain rule can be preset so that the plurality of forms of representation can be converted into a unique form of representation according to the preset rule.

[0079] For example, the first requirement label of the task submitted by the user includes at least one requirement sub-label, which may contain English capitalization. The at least one requirement sub-label can be uniformly converted to English lowercase according to the preset rule. For another example, the first requirement label includes a sub-label set formed by a plurality of requirement sub-labels, and the plurality of requirement sub-labels may not be arranged in dictionary order. The plurality of requirement sub-labels can be rearranged in dictionary order according to the preset rule. For another example, the first requirement label is an infix expression including a plurality of requirement sub-labels. The infix expression can be converted into a postfix expression according to the preset rule. The postfix expression does not contain English capitalization, and the plurality of leaf nodes of each parent node in the tree structure corresponding to the postfix expression are arranged in dictionary order. The above rules can be used alone or in combination according to actual needs. The postfix expression is also called reverse polish or reverse polish expression, which is more conducive to computer understanding and calculation.

[0080] Figure 5 FIG. 1 is a schematic diagram of a tree structure representation of a reverse polish expression provided in an embodiment of the present application. The second requirement label represented by the reverse polish expression is ‘bigmem npu¢os ubuntu|&’, and the corresponding first requirement label is ‘bigmem&npu&(centos|ubuntu)’.

[0081] It should be understood that if the requirement label submitted by the user is directly used, for example, the requirement label submitted by the user includes n requirement sub-labels. The requirement label with the same semantics as the requirement label may have at most n!-1 kinds. For example, the requirement label submitted by the user is ‘a b c’, and the requirement label with the same semantics as the requirement label ‘a b c’ has 5 kinds, which are ‘a c b’, ‘b a c’, ‘b c a’, ‘c a b’, and ‘c b a’. If the user also uses logical operators such as ‘and’, ‘or’, ‘not’, and ‘bracket’ to represent the requirement label, the number of requirement labels with the same semantics as the requirement label submitted by the user may be more. If no uniform conversion is performed, the operation amount of label matching will be increased. Therefore, uniform label conversion according to the preset rule can greatly reduce the operation amount and improve the cache hit rate.

[0082] Based on the above introduction, the management node converts the first requirement label into a second requirement label according to the preset rule. The second requirement label and the first requirement label are requirement labels with the same semantics but different representations. If the second requirement label is recorded in the historical resource allocation information, the management node determines that the historical resource allocation information records the historical resource allocation demand which is the same as the resource allocation demand of the task to be executed.

[0083] Exemplarily, the first requirement label is 'bigmem&NPU&(ubuntu|centos)', the preset rule indicates that the first requirement label is converted into a second requirement label expressed in a postfix expression, each English uppercase in the first requirement label is converted into an English lowercase, and multiple leaf nodes of each parent node in a tree structure corresponding to the postfix expression are arranged in a dictionary order. Then, the second requirement label is 'bigmem npu¢os ubuntu|&'.

[0084] Hereinafter, the first requirement label is expressed in a prefix expression, and the second requirement label is expressed in a postfix expression.

[0085] In the embodiment of the application, the requirement labels recorded in the historical resource allocation information are all in the form of a postfix expression, and the corresponding requirement labels are recorded according to a tree structure corresponding to the postfix expression.

[0086] It should be understood that, in the embodiment of the application, the historical resource allocation information records a corresponding relationship between resource allocation requirements and candidate execution nodes. In one implementation, the corresponding relationship includes a requirement label and an identifier of a corresponding candidate execution node, wherein the identifier of the execution node can be a node name or other unique identifier of the execution node.

[0087] Exemplarily, the identifier of a certain execution node is 'Node1'.

[0088] In another implementation, the corresponding relationship includes a requirement label and a corresponding binary string, wherein the binary string includes multiple binary numbers, the multiple binary numbers correspond to multiple execution nodes in the task scheduling system one by one, and the candidate execution node includes an execution node corresponding to a position of a binary number with a first numerical value in the corresponding binary string. In the embodiment of the application, the first numerical value is '1'. In some other embodiments, the first numerical value can also be '0'.

[0089] Exemplarily, a certain binary string recorded in the historical resource allocation information is '100010', the six binary numbers correspond to six execution nodes in the task scheduling system respectively, and the candidate execution node represented by the binary string includes the first execution node and the sixth execution node.

[0090] Optionally, the historical resource allocation information is recorded in a cache of the management node. In this way, the scheduling efficiency can be improved. That is, in the case of cache hit, the management node directly obtains the target execution node from the cache. The historical resource allocation information in which the same historical resource allocation requirement as the resource allocation requirement of the to-be-executed task is recorded represents cache hit. Of course, the historical resource allocation information can also be recorded in other storage spaces of the management node or storage spaces of other devices.

[0091] Step 403: When it is determined that the historical resource allocation information has the same historical resource allocation requirement as the resource allocation requirement of the to-be-executed task, the execution node recorded in the historical resource allocation information that matches the historical resource allocation requirement is determined as the target execution node.

[0092] That is, if the historical resource allocation information has the same historical resource allocation requirement as the resource allocation requirement of the to-be-executed task, the management node determines the first candidate execution node recorded in the historical resource allocation information as the target execution node. The first candidate execution node refers to the candidate execution node corresponding to the historical resource allocation requirement that is the same as the resource allocation requirement of the to-be-executed task. In the case where the resource allocation requirement of the to-be-executed task is represented by the first requirement label, the first candidate execution node refers to the candidate execution node corresponding to the requirement label (such as the second requirement label in the foregoing) recorded in the historical resource allocation information that has the same semantic as the first requirement label.

[0093] When it is determined that the historical resource allocation information has the second requirement label, the management node determines that the historical resource allocation information has the same historical resource allocation requirement as the resource allocation requirement of the to-be-executed task, and determines the first candidate execution node as the target execution node. In this way, it is not necessary to match the candidate execution node for each to-be-executed task from a plurality of execution nodes, which can improve the scheduling efficiency.

[0094] Step 404: The to-be-executed task is scheduled to the target execution node for execution.

[0095] If the target execution node includes one execution node, the management node directly schedules the to-be-executed task to the execution node. If the target execution node includes at least two execution nodes, the management node determines at least one execution node from the target execution node according to a specified strategy, and schedules the to-be-executed task to the at least one execution node. The specified strategy is a random selection strategy, that is, the management node randomly selects at least one execution node from the target execution node. Alternatively, the specified strategy is a load balancing strategy, that is, the management node selects at least one execution node from the target execution node according to the load balancing strategy. Alternatively, the specified strategy can also be other strategies, which are not limited by the embodiments of the present application.

[0096] In addition, the number of the at least one execution node can be determined according to the task quantity of the to-be-executed task or the node quantity indication information of the to-be-executed task obtained by the management node, or can be determined according to a default setting of the management node, or can be determined according to other manners. The above-mentioned manners of determining the number of the execution node can be used independently or in combination.

[0097] It should be understood that if the to-be-executed task needs to be executed by one execution node, the management node can select one execution node from the target execution nodes and schedule the to-be-executed task to the selected execution node. If the to-be-executed task needs to be executed by two execution nodes (i.e., cooperative execution), the management node can select two execution nodes from the target execution nodes and schedule the to-be-executed task to the selected two execution nodes. The above-mentioned manners can be used in a similar way.

[0098] In an implementation manner, the task information of the to-be-executed task further includes node quantity indication information. For example, the node quantity indication information is a first quantity, and the first quantity refers to the number of the execution nodes required for executing the to-be-executed task. The management node can select the first quantity of execution nodes from the target execution nodes and schedule the to-be-executed task to the selected first quantity of execution nodes.

[0099] In another implementation manner, the task information of the to-be-executed task further indicates a task quantity. The greater the task quantity is, the more execution nodes are required. The management node can select a proper number of execution nodes from the target execution nodes according to the task quantity of the to-be-executed task.

[0100] If the historical resource allocation information does not record the historical resource allocation demand identical to the resource allocation demand of the to-be-executed task, for example, does not record the second demand tag, i.e., does not record the candidate execution node corresponding to the demand tag identical in semantics to the first demand tag, the management node can determine the execution node with the node tag matching the resource allocation demand of the to-be-executed task from the plurality of execution nodes, and determine the execution node with the node tag matching the resource allocation demand of the to-be-executed task in the plurality of execution nodes as the target execution node. That is, in the case that the historical resource allocation information does not record the historical resource allocation demand identical to the resource allocation demand of the to-be-executed task, for example, has not received the task with the demand tag identical in semantics to the first demand tag, the management node needs to re-match the target execution node for the to-be-executed task from the plurality of execution nodes, and cannot directly obtain the target execution node from the historical resource allocation information.

[0101] If the management node converts the first demand label into the second demand label according to the preset rule, if the second demand label is not recorded in the historical resource allocation information, the management node determines, from the plurality of execution nodes, an execution node whose node label matches the second demand label, and determines the execution node whose node label matches the second demand label in the plurality of execution nodes as the target execution node.

[0102] It should be understood that, since the semantics of the first demand label and the second demand label are the same, the execution node whose node label matches the first demand label in the plurality of execution nodes is also the execution node whose node label matches the second demand label.

[0103] As known from the foregoing, the second demand label includes at least one demand sub-label. There are many ways for the management node to determine, from the plurality of execution nodes, an execution node whose node label matches the second demand label, and one implementation manner is described in detail here.

[0104] One implementation manner for the management node to determine, from the plurality of execution nodes, an execution node whose node label matches the second demand label is as follows: obtaining, from a node bitmap, a binary string corresponding to each of the at least one demand sub-label, the node bitmap including binary strings corresponding to a plurality of reference labels, each binary string including a plurality of binary numbers corresponding to the plurality of execution nodes in the task scheduling system in one-to-one manner, each binary number indicating whether the corresponding execution node has the corresponding reference label; replacing the at least one demand sub-label in the target postfix expression with the corresponding binary string to obtain a to-be-operated expression, the target postfix expression being a postfix expression used to represent the second demand label; determining the value of the to-be-operated expression through bit operation to obtain a target binary string; and determining, as the execution node whose node label matches the second demand label in the plurality of execution nodes, an execution node having the corresponding reference label in the target binary string.

[0105] In an example, the task scheduling system includes five execution nodes, and the node labels of the five execution nodes are as follows.

[0106] 0th node: bigmem, npu, ubuntu

[0107] 1st node: bitmem, centos

[0108] 2nd node: npu, ubuntu

[0109] 3rd node: bigmem

[0110] 4th node: centos

[0111] The reference tags in the example include four, namely 'bigmem', 'npu', 'ubuntu' and 'centos', and the node bitmap formed by the above 5 execution nodes includes four binary strings corresponding to the four reference tags, which are '11010', '10100', '01001' and '10100' respectively. The relationship between the reference tags and the node bitmap is shown in Table 1.

[0112] Table 1

[0113]

[0114] Taking the first demand tag 'bigmem&NPU&(centos|ubuntu)' and the second demand tag 'bigmemnpu¢os ubuntu|&' as an example, the target suffix expression is 11010 10100&01001 10100|&, and the value of the target suffix expression can be obtained by bit operation as 10000. Since the target suffix expression has the same semantics as the infix expression representing the first demand tag, in order to facilitate the reader to understand, the calculation method of the infix expression according to human thinking is used here to explain the above operation result, that is, at least one demand sub-tag in the infix expression is replaced by the corresponding binary string to obtain an expression with the same semantics as the above to-be-operated expression, that is, bigmem&npu&(centos|ubuntu) is replaced by the expression 11010&10100&(01001|10100), and the expression is obtained by bit operation as 10000, which is also the value of the above to-be-operated expression. That is, the obtained target binary string is '10000', and only the first bit of the target binary string has a value of '1', indicating that only the first bit has the corresponding reference tag, so the management node determines the 0th execution node corresponding to the first bit as the execution node whose node tag matches the second demand tag among the above 5 execution nodes, and determines the 0th execution node as the target execution node.

[0115] In the above example, the position with a value of 1 in the node bitmap indicates that the corresponding execution node has the corresponding reference tag, and the position with a value of 0 indicates that the corresponding execution node does not have the corresponding reference tag.

[0116] It should be understood that the above example is only used to explain the present scheme and does not limit the present scheme, and in specific implementation, the reference tags can also include 'arm', 'x86', 'aarch64' and the like.

[0117] As can be seen from the above examples, compared with the string comparison manner, the embodiments of the present application can greatly reduce the calculation amount by using the tree structure reverse polish (i.e., postfix expression) and bit operation, thereby improving the scheduling efficiency and task execution efficiency.

[0118] Of course, in some other embodiments, such as in some scenarios where the efficiency requirement is not particularly high, such as in some scenarios where the computing capability of the management node is strong, the string comparison manner can also be used to determine the execution node with the matching node label and second demand label from the plurality of execution nodes.

[0119] In the embodiments of the present application, after the execution node with the matching node label and first demand label in the plurality of execution nodes is determined as the target execution node, the corresponding relationship between the first demand label and the target execution node is added in the historical resource allocation information.

[0120] In the embodiment where the management node converts the first demand label into the second demand label, the management node adds the corresponding relationship between the second demand label and the identifier of the target execution node in the historical resource allocation information, or adds the corresponding relationship between the second demand label and the target binary string in the historical resource allocation information.

[0121] By recording the binary string in the historical resource allocation information to represent the corresponding candidate execution node, the amount of data stored can be reduced. In the case where the historical resource allocation information is recorded in the cache of the management node, the waste of the cache can also be reduced.

[0122] It should be understood that the demand label and the binary string recorded in the historical resource allocation information can be regarded as a hash key-value pair, wherein the demand label is regarded as the 'key', and the binary string is regarded as the 'hash value'. By caching the hash key-value pair, the cache hit rate can be greatly improved, and the memory occupation can be reduced.

[0123] Optionally, after obtaining the target binary string, the management node directly adds the corresponding relationship between the second demand label and the target binary string in the cached historical resource allocation information. Alternatively, after obtaining the target binary string, the management node judges whether the corresponding relationship between the second demand label and the target binary string satisfies the cache entry condition. If the corresponding relationship satisfies the cache entry condition, the management node adds the corresponding relationship between the second demand label and the target binary string in the cached historical resource allocation information. If the corresponding relationship does not satisfy the cache entry condition, the management node does not add the corresponding relationship between the second demand label and the target binary string in the cached historical resource allocation information.

[0124] The cache entry condition can include a resource allocation demand recorded in the historical resource allocation information, which includes a resource allocation demand whose number of sub-demands exceeds a first threshold, and / or a resource allocation demand whose number of demands exceeds a second threshold, and can also include other conditions. In this way, when the number of demand sub-labels included in the second demand label does not exceed the first threshold, and / or the number of demands of the demand sub-labels included in the second demand label exceeds the second threshold, the management node adds the correspondence between the second demand label and the target binary string in the historical resource allocation information.

[0125] The first threshold can be 3 or 5 or other values. The smaller the first threshold, the less the corresponding task requires the execution node, and users usually do not set many demand sub-labels, that is, the fewer the demand sub-labels of a task, the closer the task is to a popular demand task, and the cache stores as many candidate execution nodes matched with popular demand tasks as possible, thereby providing cache hit rate. That is, by limiting the first threshold, the cache hit rate can be improved. Similarly, the second threshold can be 10, 15, or other values. The larger the second threshold, the higher the corresponding resource demand, and the cache demand high information can improve the cache hit rate.

[0126] In addition, due to the limited cache space, the management node can evict obsolete caches in the historical resource allocation information according to the least recently used (LRU) principle or other principles to provide cache space for new correspondences.

[0127] In the embodiments of the present application, the management node generates a node bitmap based on the node labels of each execution node in the task scheduling system and the plurality of reference labels before using the node bitmap to determine the to-be-operated expression. The generated node bitmap represents whether each execution node in the plurality of execution nodes has each reference label in the plurality of reference labels.

[0128] Before generating the node bitmap, the management node first acquires the node label of each execution node in the plurality of execution nodes. There are various ways to acquire the node label, such as manually labeling the execution node by the administrator and / or automatically labeling the execution node by the system.

[0129] In the manner that the administrator manually labels the execution nodes, the administrator configures node labels for one or at least two execution nodes through a command line tool or a visual operation interface, wherein one or more node sub-labels can be configured for each execution node, and the configuration information is submitted to the management node. For example, the administrator configures at least two node sub-labels for a first execution node in the plurality of execution nodes, generates first configuration information through the command line tool or the visual operation interface, and submits the first configuration information to the management node. For another example, the administrator configures node labels for at least two execution nodes in the plurality of execution nodes, and submits second configuration information to the management node.

[0130] For example, the administrator labels the execution nodes using the command line tool as follows.

[0131] / / Label node1 and node3 with bigmem and npu

[0132] node_cli –-lables ‘bigmem,npu’ node1 node3

[0133] / / Label node1, node2 and node4 with gpu

[0134] node_cli-l ‘gpu’ node1 node2 node4

[0135] Correspondingly, the management node receives the first configuration information, the first configuration information including at least two node sub-labels configured for a first execution node in the plurality of execution nodes, the first execution node being any one execution node including a plurality of node sub-labels in a node label; and / or the management node receives the second configuration information, the second configuration information including node labels configured for at least two execution nodes in the plurality of execution nodes.

[0136] It can be seen that the present scheme supports configuring multiple node sub-labels for one execution node at a time, and also supports configuring node labels for a batch of execution nodes at a time. In this way, the flexibility of the administrator in configuring custom labels can be improved, and the ease of use and the functional completeness of the task scheduling system can be improved.

[0137] In the manner that the system automatically labels the execution nodes, the execution nodes actively report their node information, or send the node information of the execution nodes to the management node after receiving a request from the management node, and the management node automatically obtains information that can be used as node sub-labels from the node information.

[0138] For example, the node information reported by the first execution node includes ‘OS is centos and CPU architecture is x86_64’, and then the management node can automatically set the node label of the first execution node to include ‘centos and x86_64’.

[0139] It should be understood that, in order to ensure that the custom label configured by the administrator does not conflict with the label automatically punched by the system, in the case that the first node sub-label in the configuration information received by the management node conflicts with the node sub-label automatically punched by the system, the management node sends a prompt information to the client used by the administrator, which indicates that the first node sub-label configured by the administrator conflicts with the node sub-label automatically punched by the system.

[0140] Next, please refer to Figure 6 The implementation process of the management node obtaining the node label and generating the node bitmap in the embodiments of the present application is further exemplarily explained.

[0141] Figure 6 is a method flowchart for obtaining a node label and generating a node bitmap provided by the embodiments of the present application. Taking the management node including Figure 2 the resource management interface, the resource management module and the job scheduling module as shown in the figure, referring to Figure 6 , the resource management module receives the node label punched by the administrator (i.e. receives the configuration information) through the management interface, and receives the node label reported by the execution node (i.e. the system automatically punches the label), based on the node label of the execution node, constructs the binary string corresponding to the corresponding reference label, i.e. constructs the bitmap of the corresponding reference label, and constructs the global node bitmap based on the bitmaps of all reference labels. The resource management module synchronizes the node bitmap to the job scheduling module, i.e. the node bitmap is synchronized to the scheduling engine (i.e. the job scheduling module), so as to facilitate the job scheduling module to match the target execution node based on the node bitmap for the to-be-executed task.

[0142] Next, please refer to Figure 7 An implementation manner of the task scheduling method provided by the embodiments of the present application is exemplarily explained and described.

[0143] Figure 7 is a flowchart of another task scheduling method provided by the embodiments of the present application. Referring to Figure 7 , the management node receives the task submitted by the user through the job submission interface, and the task information of the task includes a demand label expressed in the infix expression. The management node parses the infix expression through the resource management module, constructs the tree structure of the reverse polish expression corresponding to the infix expression, and assigns the task to the job scheduling module in the management node, i.e. assigns the task to the scheduling engine, including assigning the constructed tree structure.

[0144] The job scheduling module selects a task from the tasks to be executed for scheduling. The job scheduling module determines to enter a label requirement filtering stage, in which the job scheduling module queries from the historical resource allocation information cached by the management node whether there is a candidate execution node corresponding to a requirement label with the same requirement label semantics as the task to be executed. In brief, in this stage, the cache is queried first. If the cache hits, the job scheduling module directly obtains the candidate execution node corresponding to the requirement label with the same requirement label semantics as the task to be executed, that is, directly obtains the candidate execution node satisfying the label requirement.

[0145] If the cache misses, the job scheduling module determines the target binary string based on the node bitmap and the above reverse Polish expression through bit operation, that is, calculates the candidate execution node satisfying the label requirement. For the specific implementation process, please refer to the related content above, which is not repeated here. After determining the target binary string, the job scheduling module determines the target execution node based on the target binary string. For the specific implementation process, please refer to the related content above. At this point, the job scheduling module has calculated the node set satisfying the requirement label requirement. The execution nodes in the node set are the target execution nodes, which can include one execution node or at least two execution nodes.

[0146] After determining the target execution node, the job scheduling module performs the subsequent scheduling process based on the target execution node, for example, determines at least one execution node from the target execution node according to a specified strategy, and schedules the task to be executed to the at least one execution node.

[0147] In addition, the job scheduling module also judges whether the correspondence between the above reverse Polish expression and the target binary string satisfies the cache entry condition. In the case where the correspondence satisfies the cache entry condition, the job scheduling module adds the correspondence between the reverse Polish expression and the target binary string in the historical resource allocation information, that is, enters the cache. In addition, since the cache space is limited, the job scheduling module can eliminate the obsolete cache in the historical resource allocation information according to the LRU principle, so as to provide cache space for the new correspondence.

[0148] In summary, in the embodiment of the present application, after obtaining the resource allocation requirement of the task to be executed, if the historical resource allocation information records a historical resource allocation requirement same as the resource allocation requirement of the task to be executed, the execution node matching the historical resource allocation requirement recorded in the historical resource allocation information is directly determined as the target execution node. In this way, it is not necessary to determine the execution node with the node label matching the resource allocation requirement of the task to be executed from a plurality of execution nodes again, and a large amount of string matching calculation is not necessary, which improves the screening efficiency of the execution node and thus improves the scheduling efficiency and the execution efficiency of the task.

[0149] In a specific implementation, a user can use AND NOT logic syntax to describe resource allocation requirements of a task, and an administrator can customize tags for execution nodes, apply multiple node sub-tags to one execution node at a time, and apply tags to multiple execution nodes in sequence (i.e., set operation). The management node uses bit operation and reverse Polish notation to convert tag scheduling into efficient bit operation and improve performance. Hash key-value pairs are constructed to cache scheduling results, and reverse Polish notation is used to combine tag requirements of the same semantics, thereby improving cache hit rate.

[0150] Figure 8 FIG. 8 is a structural schematic diagram of a task scheduling apparatus 800 provided by an embodiment of the present application. The task scheduling apparatus 800 can be implemented as part or all of a computer device by software, hardware, or a combination of both, and the computer device can be a management node as shown in FIG. 1 or FIG. 2. Figure 1 or Figure 2 In an embodiment of the present application, the apparatus 800 is included in a management node in a task scheduling system, and the task scheduling system further includes multiple execution nodes. Referring to FIG. 8, the apparatus 800 includes a first obtaining module 801, a first determining module 802, a second determining module 803, and a scheduling module 804. Figure 8

[0151] The first obtaining module 801 is configured to obtain resource allocation requirements of a task to be executed.

[0152] The first determining module 802 is configured to determine whether there is a historical resource allocation requirement that is the same as the resource allocation requirements of the task to be executed in historical resource allocation information recorded by the management node. The historical resource allocation information is used to record information of an execution node matched by a resource allocation requirement of a task that has been executed.

[0153] The second determining module 803 is configured to determine, when it is determined that there is a historical resource allocation requirement that is the same as the resource allocation requirements of the task to be executed in the historical resource allocation information, an execution node recorded in the historical resource allocation information and matched by a historical resource allocation requirement as a target execution node.

[0154] The scheduling module 804 is configured to schedule the task to be executed to the target execution node for execution.

[0155] Optionally, the resource allocation requirements include multiple sub-requirements, and the multiple sub-requirements are expressed in a manner of a logical expression.

[0156] Optionally, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of sub-requirements exceeds a first threshold value.

[0157] Optionally, the historical resource allocation requirements recorded in the historical resource allocation information include resource allocation requirements whose number of requirements exceeds a second threshold value. ​

[0158] Optionally, the resource allocation requirement of the task to be executed is represented by a first requirement label; the apparatus 800 further comprises:

[0159] a label conversion module, configured to convert the first requirement label into a second requirement label according to a preset rule, the second requirement label being a requirement label that is semantically same as the first requirement label but has different representation forms;

[0160] a first determination module, specifically configured to determine that the historical resource allocation information records a historical resource allocation requirement same as the resource allocation requirement of the task to be executed if the second requirement label is recorded in the historical resource allocation information.

[0161] Optionally, the apparatus 800 further comprises:

[0162] a requirement matching module, configured to determine, if the second requirement label is not recorded in the historical resource allocation information, an execution node whose node label matches the second requirement label from the plurality of execution nodes;

[0163] a third determination module, configured to determine the execution node whose node label matches the second requirement label from the plurality of execution nodes as the target execution node.

[0164] Optionally, the first requirement label is represented in the form of an infix expression, and the second requirement label is represented in the form of a postfix expression.

[0165] Optionally, the second requirement label comprises at least one requirement sub-label.

[0166] The requirement matching module is specifically configured to:

[0167] obtain, from a node bitmap, a binary string corresponding to each requirement sub-label, the node bitmap comprising binary strings corresponding to a plurality of reference labels, each binary string comprising a plurality of binary digits corresponding to the plurality of execution nodes, each binary digit indicating whether a corresponding execution node has a corresponding reference label;

[0168] replace the at least one requirement sub-label in the target postfix expression with a corresponding binary string to obtain a to-be-operated expression, the target postfix expression being a postfix expression used to represent the second requirement label;

[0169] determine a value of the to-be-operated expression through bit operation to obtain a target binary string;

[0170] determine, from the target binary string, an execution node having a corresponding reference label as the execution node whose node label matches the second requirement label from the plurality of execution nodes.

[0171] Optionally, the apparatus 800 further comprises:

[0172] Add a module to add a mapping between the second requirement tag and the target binary string in the historical resource allocation information.

[0173] Optionally, the device 800 further includes:

[0174] The second acquisition module is used to acquire the node label of each execution node among the multiple execution nodes;

[0175] The generation module is used to generate a node bitmap based on the node labels of each execution node and the multiple reference labels. The node bitmap represents whether each execution node has each of the multiple reference labels.

[0176] Optionally, the second acquisition module is specifically used for:

[0177] Receive first configuration information, the first configuration information including at least two node sub-labels configured for a first execution node among the plurality of execution nodes, the first execution node being any one of the execution nodes whose node label includes the plurality of node sub-labels; and / or,

[0178] Receive second configuration information, which includes node labels configured for at least two of the plurality of execution nodes.

[0179] Optionally, historical resource allocation information is recorded in the cache of the management node.

[0180] In this embodiment, after obtaining the resource allocation requirements of the task to be executed, if the historical resource allocation information records a historical resource allocation requirement that is identical to the resource allocation requirement of the task to be executed, then the execution node that matches the historical resource allocation requirement recorded in the historical resource allocation information is directly determined as the target execution node. This eliminates the need to re-determine the execution node whose node label matches the resource allocation requirement of the task to be executed from multiple execution nodes, avoiding extensive string matching calculations, thus improving the efficiency of execution node selection, and consequently improving scheduling efficiency and task execution efficiency.

[0181] It should be noted that the task scheduling device provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the task scheduling device and the task scheduling method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0182] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, the computer instructions generate all or part of the processes or functions described in the embodiments of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another via wired (for example: coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example: infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer, or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example: floppy disk, hard disk, magnetic tape), optical media (for example: digital versatile disc (DVD)) or semiconductor media (for example: solid state disk (SSD)) etc. It should be noted that the computer-readable storage medium mentioned in the embodiments of the present application can be a non-volatile storage medium, in other words, it can be a non-transitory storage medium.

[0183] It should be understood that "at least one" mentioned herein refers to one or more, and "multiple" refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, "first", "second" and the like are used to distinguish the same or similar items with basically the same function and role in the embodiments of the present application. Those skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and "first", "second" and the like do not necessarily mean different.

[0184] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0185] The above describes the embodiments provided by the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A task scheduling method, characterized in that, The method is applied to a management node in a task scheduling system, which further includes multiple execution nodes, and includes: Obtain the resource allocation requirements of the task to be executed, wherein the resource allocation requirements of the task to be executed are represented by a first requirement tag; If the historical resource allocation information recorded by the management node contains a second demand tag with the same semantics as the first demand tag, then the execution node with the corresponding reference tag in the target binary string is determined as the target execution node. The historical resource allocation information is used to record the information of the execution node that matches the resource allocation demand of the executed task by the management node. The historical resource allocation information includes the correspondence between the second demand tag and the binary string. Each binary string includes multiple binary numbers that correspond one-to-one with the plurality of execution nodes. Each binary number indicates whether the corresponding execution node has a corresponding reference tag. The target binary string is the binary string corresponding to the second demand tag with the same semantics as the first demand tag in the historical resource allocation information. The task to be executed is scheduled to the target execution node for execution.

2. The method as described in claim 1, characterized in that, The resource allocation requirement includes multiple sub-requirements, which are represented by logical expressions.

3. The method as described in claim 1, characterized in that, The historical resource allocation information records historical resource allocation needs, including resource allocation needs where the number of sub-needs exceeds a first threshold.

4. The method as described in claim 1, characterized in that, The historical resource allocation information records historical resource allocation requests, including resource allocation requests whose frequency exceeds the second threshold.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The first requirement tag is converted into a second requirement tag according to a preset rule. The second requirement tag is a requirement tag with the same semantics as the first requirement tag but with a different representation. If the historical resource allocation information records the second demand tag, then it is determined that the historical resource allocation information records a demand tag with the same semantics as the first demand tag.

6. The method as described in claim 5, characterized in that, The method further includes: If the second requirement tag is not recorded in the historical resource allocation information, then the execution node whose node tag matches the second requirement tag is determined from the plurality of execution nodes; The execution node whose node label matches the second requirement label among the plurality of execution nodes is determined as the target execution node.

7. The method as described in claim 5, characterized in that, The first requirement tag is represented using infix notation, and the second requirement tag is represented using postfix notation.

8. The method as described in claim 6, characterized in that, The second requirement tag includes at least one requirement sub-tag; The step of determining the execution node whose node label matches the second requirement label from the plurality of execution nodes includes: Obtain the binary strings corresponding to the at least one requirement sub-tag from the node bitmap, wherein the node bitmap includes binary strings corresponding to multiple reference tags; Replace at least one requirement sub-tag in the target postfix expression with the corresponding binary string to obtain the expression to be calculated, wherein the target postfix expression refers to the postfix expression used to represent the second requirement tag; The value of the expression to be operated on is determined by bitwise operations, thus obtaining the target binary string; The execution node with the corresponding reference label in the target binary string is determined as the execution node whose node label matches the second requirement label among the plurality of execution nodes.

9. The method as described in claim 8, characterized in that, The method further includes: Add the correspondence between the second demand tag and the target binary string to the historical resource allocation information.

10. The method as described in claim 8 or 9, characterized in that, The method further includes: Obtain the node label of each execution node among the plurality of execution nodes; Based on the node labels of each execution node in the plurality of execution nodes and the plurality of reference labels, the node bitmap is generated, and the node bitmap represents whether each execution node in the plurality of execution nodes has each of the reference labels in the plurality of reference labels.

11. The method as described in claim 10, characterized in that, The step of obtaining the node label of each execution node among the plurality of execution nodes includes: Receive first configuration information, the first configuration information including at least two node sub-labels configured for a first execution node among the plurality of execution nodes, wherein the first execution node is any one of the execution nodes whose node label includes the plurality of node sub-labels; and / or, Receive second configuration information, the second configuration information including node labels configured for at least two of the plurality of execution nodes.

12. The method according to any one of claims 1-4, 6-9, and 11, characterized in that, The historical resource allocation information is recorded in the cache of the management node.

13. A task scheduling device, characterized in that, The device includes a management node in a task scheduling system, which also includes multiple execution nodes, and the device comprises: The first acquisition module is used to acquire the resource allocation requirements of the task to be executed, wherein the resource allocation requirements of the task to be executed are represented by a first requirement tag; The first determining module is used to confirm whether the historical resource allocation information recorded by the management node contains a second demand tag with the same semantics as the first demand tag. The historical resource allocation information is used to record the information of the execution node that matches the resource allocation demand of the executed task by the management node. The historical resource allocation information includes the correspondence between the second demand tag and binary strings. Each binary string includes multiple binary numbers that correspond one-to-one with the plurality of execution nodes. Each binary number indicates whether the corresponding execution node has a corresponding reference tag. The second determining module is used to determine the execution node with the corresponding reference tag in the target binary string as the target execution node when it is determined that there is a second demand tag in the historical resource allocation information that has the same semantics as the first demand tag. The target binary string is the binary string corresponding to the second demand tag in the historical resource allocation information that has the same semantics as the first demand tag. The scheduling module is used to schedule the task to be executed to the target execution node for execution.

14. The apparatus as claimed in claim 13, characterized in that, The historical resource allocation information records historical resource allocation needs, including resource allocation needs where the number of sub-needs exceeds a first threshold.

15. The apparatus as claimed in claim 13 or 14, characterized in that, The device further includes: The tag conversion module is used to convert the first requirement tag into a second requirement tag according to a preset rule. The second requirement tag is a requirement tag with the same semantics as the first requirement tag but with a different representation. The first determining module is specifically used to determine that if the historical resource allocation information records the second demand tag, then the historical resource allocation information records a demand tag with the same semantics as the first demand tag.

16. The apparatus as claimed in claim 15, characterized in that, The device further includes: The demand matching module is used to determine, from the plurality of execution nodes, the execution node whose node label matches the second demand label if the second demand label is not recorded in the historical resource allocation information; The third determining module is used to determine the execution node whose node label matches the second requirement label among the plurality of execution nodes as the target execution node.

17. The apparatus as claimed in claim 15, characterized in that, The first requirement tag is represented using infix notation, and the second requirement tag is represented using postfix notation.

18. A task scheduling system, characterized in that, The task scheduling system includes a management node and multiple execution nodes; The management node is used to execute the steps of the method according to any one of claims 1-12.

19. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-12.

20. A computer program product, characterized in that, The computer program product stores computer instructions, which, when executed by a processor, implement the method described in any one of claims 1-12.

Images