Resource scheduling method and device

Abstract

The invention provides a resource scheduling method and device, relates to the field of computers, and is used for improving the resource utilization rate of computing nodes. The method comprises the following steps: a computing node obtains a resource allocation request; the resource allocation request is used for requesting to allocate a container for the target task in the computing node, and the resource allocation request comprises task content corresponding to the target task. The computing node preloads the task content in the first container and determines the memory occupied by the first container; wherein the size of the memory occupied by the first container corresponds to that of the memory occupied by the preloading task content. And after the task content is preloaded, the computing node determines to allocate the first container for the target task.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a resource scheduling method and apparatus. Background Technology

[0002] Currently, the resources of a computing node can be divided into multiple containers. Each container can be understood as an independent operating environment that encapsulates the various resources required for task execution (such as memory, central processing unit (CPU), disk, and network). Therefore, in the process of allocating resources for tasks, containers can be used as the abstract unit of resources for resource scheduling.

[0003] For example, such as Figure 1 The diagram shows a schematic of a computing node. The resources within this computing node 10 can be pre-divided into multiple containers (as shown in container 11, container 12... container 1n). During resource scheduling, containers within the computing node can be allocated to applications according to actual needs, so that corresponding tasks can run within the containers.

[0004] Among these issues, improving the resource utilization efficiency of computing nodes during resource scheduling is a problem that needs to be solved. Summary of the Invention

[0005] This application provides a resource scheduling method and apparatus for improving the resource utilization of computing nodes.

[0006] Firstly, a resource scheduling method is provided, comprising: a computing node acquiring a resource allocation request; the resource allocation request is used to request the allocation of a container for a target task in the computing node, the resource allocation request including the task content corresponding to the target task; the computing node preloads the task content in a first container and determines the memory occupied by the first container; wherein the memory size occupied by the first container corresponds to the memory occupied by the preloaded task content; after the task content is preloaded, the computing node determines to allocate the first container for the target task.

[0007] In the method, when the computing node obtains a resource allocation request for requesting to allocate a container for a task (i.e., a "target task"), the computing node can first preload task content corresponding to the target task in a container (i.e., a "first container") in the computing node, and determine memory occupied by the first container. The memory occupied by the first container corresponds to memory occupied by the preloaded task content. Then, after the task content is preloaded, the computing node determines to allocate the first container for the target task. In this way, after the first container is allocated to the target task, the target task can be ensured to normally run in the first container, and waste of memory resources is avoided, thereby improving resource use efficiency of the computing node.

[0008] In an implementation, the method further includes: the computing node detecting whether the task content meets preset conditions; the preset conditions include one or more of the following: a file format of the task content meets a preset format, a type of service data of the task content meets a preset type, and a data amount corresponding to the task content meets a preset data amount. The computing node preloading the task content in the first container includes: the computing node preloading the task content in the first container after determining that the task content meets the preset conditions.

[0009] In the implementation, it is considered that there can be some task content of a task that is not suitable for preloading. Therefore, when the task content is preloaded, it is first determined whether the task content meets preset conditions. The preloading is performed after it is determined that the task content meets the preset conditions. In this way, the running efficiency of the task is improved.

[0010] In an implementation, the method further includes: the computing node receiving a first configuration instruction. The first configuration instruction is used to determine the preset conditions.

[0011] In the implementation, the first configuration instruction can be used to set the corresponding preset conditions. Therefore, different contents of the first configuration instruction can be sent to the computing node according to actual needs, so that the corresponding preset conditions are obtained.

[0012] In an implementation, the computing node preloading the task content in the first container includes: the computing node preloading the task content in the first container after determining that a resource preloading function is turned on. The method further includes: the computing node determining to allocate a second container for the target task after determining that the resource preloading function is turned off, the second container occupying a preset memory size.

[0013] By the implementation, the container of the task can be determined by preloading the task content when the resource preloading function is enabled. When the resource preloading function is disabled, the container of the task is determined by allocating the container with a preset memory size, instead of preloading the task content. In this way, whether to enable the resource preloading function can be determined according to actual application requirements.

[0014] In an implementation, the method further includes that the computing node receives a second configuration instruction. The second configuration instruction is used to enable or disable the resource preloading function.

[0015] By the implementation, the second configuration instruction can be used to control the enabling or disabling of the resource preloading function.

[0016] In an implementation, the method further includes that after the preloading of the task content is completed, the computing node sets the first container to a to-be-pulled state. The to-be-pulled state is used to indicate that the first container has been occupied.

[0017] In the implementation, after the preloading of the task content is completed, the first container is set to the to-be-pulled state, so that the first container can be prevented from being occupied by other applications before the application manager issues the target task to the first container for running.

[0018] In an implementation, the method further includes that the computing node receives a running instruction from the application manager. The running instruction is used to indicate that the target task is run in the first container. The computing node runs the target task in the first container by using the preloaded task content.

[0019] In the implementation, after the computing node receives the running instruction from the application manager, the computing node can run the target task in the first container by using the preloaded task content. In this way, the running time of each container can be reduced, and the entire application execution time can be reduced.

[0020] In an implementation, the computing node is a computing node in a distributed system, and the resource allocation request is sent to the computing node by a resource manager in the distributed system. After the preloading of the task content is completed, the computing node determines to allocate the first container for the target task, including that the computing node sends resource information to the resource manager after the preloading of the task content is completed. The resource information is used to indicate that the first container is allocated for the target task.

[0021] In a second aspect, a data processing apparatus is provided, which is applied to a computing node, and includes: a communication unit configured to obtain a resource allocation request, the resource allocation request being used to request allocation of a container for a target task in the computing node, and the resource allocation request carrying task content corresponding to the target task; and a processing unit configured to preload the task content in a first container, and determine memory occupied by the first container, wherein the memory occupied by the first container corresponds to memory occupied by the preloaded task content; and the communication unit is further configured to determine allocation of the first container for the target task after the task content is preloaded.

[0022] In an implementation manner, the processing unit is further configured to detect whether the task content meets preset conditions, the preset conditions including one or more of the following: a file format of the task content meets a preset format, a type of service data of the task content meets a preset type, and a data volume corresponding to the task content meets a preset data volume; and the processing unit is configured to preload the task content in the first container, including: the processing unit is configured to preload the task content in the first container after determining that the task content meets the preset conditions.

[0023] In an implementation manner, the communication unit is further configured to receive a first configuration instruction, the first configuration instruction being used to determine the preset conditions.

[0024] In an implementation manner, the processing unit is configured to preload the task content in the first container, including: the processing unit is configured to preload the task content in the first container after determining that a resource preloading function is enabled; and the processing unit is further configured to determine allocation of a second container for the target task after determining that the resource preloading function is disabled, the second container occupying a preset memory size.

[0025] In an implementation manner, the communication unit is further configured to receive a second configuration instruction, the second configuration instruction being used to enable or disable the resource preloading function.

[0026] In an implementation manner, the processing unit is further configured to set the first container to a standby state after the task content is preloaded, the standby state being used to indicate that the first container has been occupied.

[0027] In an implementation manner, the communication unit is further configured to receive a running instruction from an application manager, the running instruction being used to instruct running of the target task in the first container; and the processing unit is further configured to run the target task in the first container by using the preloaded task content.

[0028] In an implementation manner, the computing node is a computing node in a distributed system, and the resource allocation request is sent by a resource manager in the distributed system to the computing node; the processing unit is configured to determine to allocate the first container for the target task after the task content preloading is completed, including: the processing unit is configured to send resource information to the resource manager after the task content preloading is completed, the resource information being used to indicate to allocate the first container for the target task.

[0029] In a third aspect, a data processing apparatus is provided, including a processor and an interface, the processor receiving or sending data through the interface, and the processor being configured to implement the method according to the first aspect or any implementation manner of the first aspect.

[0030] In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium storing instructions, when the instructions are executed on a processor, implementing the method according to the first aspect or any implementation manner of the first aspect.

[0031] In a fifth aspect, a computer program product is provided, the computer program product including instructions, when the instructions are executed on a processor, implementing the method according to the first aspect or any implementation manner of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A structure schematic diagram of a computing node provided by an embodiment of the present application;

[0033] Figure 2 A structure schematic diagram of a distributed system provided by an embodiment of the present application;

[0034] Figure 3 A flow schematic diagram of a resource scheduling method provided by an embodiment of the present application;

[0035] Figure 4 A flow schematic diagram of a resource scheduling method provided by an embodiment of the present application;

[0036] Figure 5 A flow schematic diagram of a resource scheduling method provided by an embodiment of the present application;

[0037] Figure 6 A flow schematic diagram of a resource scheduling method provided by an embodiment of the present application;

[0038] Figure 7 A flow schematic diagram of a resource scheduling method provided by an embodiment of the present application;

[0039] Figure 8 A structure schematic diagram of a computing node provided by an embodiment of the present application;

[0040] Figure 9 This is one of the structural schematic diagrams of a data processing device provided in an embodiment of this application;

[0041] Figure 10 This is one of the structural schematic diagrams of a data processing device provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions provided in the embodiments of this application will be described below with reference to specific examples.

[0043] Currently, to allocate computing node resources to different tasks, related technologies pre-divide the computing node resources into multiple containers. A container can be understood as an independent operating environment that encapsulates the various resources required for task execution (such as memory, central processing unit (CPU), disk, and network). Then, during the resource allocation process for tasks, containers can be used as the abstract unit for resource scheduling.

[0044] For example, such as Figure 1 The diagram shows a schematic of a computing node. The resources within this computing node 10 can be pre-divided into multiple containers (as shown in container 11, container 12... container 1n). During resource scheduling, containers within the computing node can be allocated to applications according to actual needs, so that corresponding tasks can run within the containers.

[0045] In view of the above-mentioned related technologies, the embodiments of this application take into account that as the CPU resources of computing nodes in a distributed system increase, the resource scheduling scheme based on the above-mentioned related technologies may lead to insufficient utilization of computing node resources due to the limitation of the memory capacity of computing nodes.

[0046] by Figure 1 Taking compute node 10 as an example, assume that compute node 10 includes 256 CPU cores and 512GB of memory. On the one hand, if containers are divided according to the method of allocating 1 CPU core and 4GB of memory to each container, 128 CPU cores will be idle. On the other hand, if containers are divided according to the method of allocating 1 CPU core and 2GB of memory to each container, although the 256 CPU cores can be fully utilized, some tasks may encounter insufficient memory during operation.

[0047] Based on the above problems, the embodiment of the present application provides a resource scheduling method. In the method, when a computing node obtains a resource allocation request for requesting to allocate a container for a task (hereinafter referred to as a target task), the target task corresponding task content can be preloaded in a container (hereinafter referred to as a first container) in the computing node, and the memory occupied by the first container is determined. The memory size occupied by the first container corresponds to the memory occupied by the preloaded task content. Then, after the task content is preloaded, the computing node determines to allocate the first container for the target task. In this way, after the first container is allocated to the target task, the target task can be ensured to run normally in the first container, and the waste of memory resources is avoided. In addition, the use efficiency of CPU resources in the computing node can be improved by allocating idle CPU resources to the first container, so that the resource use efficiency of the computing node is improved.

[0048] The specific implementation process of the resource scheduling method provided by the embodiment of the present application will be described in detail below with examples. Specifically, the resource scheduling method provided by the embodiment of the present application can be used in a distributed system. For example, taking a Hadoop (a distributed system infrastructure developed by Apache Foundation) system as an example, as shown in FIG. 1, a structure schematic diagram of a distributed system provided by the embodiment of the present application is shown. Figure 2

[0049] The distributed system 20 can include a plurality of computing nodes (three computing nodes are taken as an example in the figure). A plurality of containers can be divided in each computing node. In the resource scheduling process, one or more containers in the computing node can be allocated to an application according to the application requirement, so as to run corresponding tasks in the containers.

[0050] In addition, the distributed system 20 further includes a resource management system for resource management. For example, the resource management system can be yet another resource negotiator (YARN).

[0051] Specifically, the resource management system can include a resource manager (RM) (for example, the resource manager 24 in FIG. 1) for managing the entire distributed system 20, a node manager (NM) (for example, the node manager 211, the node manager 221 and the node manager 231 in FIG. 1) for managing each computing node, and an application master (AM) (for example, the application master 212, the application master 222 and the application master 232 in FIG. 1) for managing a single task running. Figure 2 Figure 2 Figure 2 ​​​For example, the application manager 212 can run in a container in the computing node.

[0052] When the distributed system 20 allocates resources for the target task, the resource scheduling process can include the following steps:

[0053] S301, the application manager 212 sends an allocation request to the resource manager 24. The allocation request can include the number of containers required by the target task, the container identifier corresponding to the application manager 212, and other information. After receiving the resource allocation request, the resource manager 24 can determine the corresponding computing node (for example, the computing node 22) according to the resource allocation request.

[0054] S302, the resource manager 24 sends a resource allocation request to the node manager 221 corresponding to the computing node 22.

[0055] S303, after obtaining the resource allocation request, the node manager 221 can determine the container (assuming that the container 223 is allocated for the target task) allocated for the target task according to the method provided in the embodiments of the present application, and feed back the resource information to the resource manager 24. The resource information can include the identifier of the container 223 and other information.

[0056] S304, the resource manager 24 sends the container information to the application manager 212.

[0057] S305, the application manager 212 interacts with the node manager 221 according to the container information, so as to run the target task by using the container 223.

[0058] S306, during the process of running the target task by using the container 223, the node manager 221 can feed back the running state of the container 223 and other information to the application manager 212.

[0059] Next, the process of allocating resources for the target task in the computing node 22 in the distributed system 20 will be taken as an example to introduce the resource scheduling method provided in the embodiments of the present application. Specifically, as shown in Figure 3 The method can include:

[0060] S401, the computing node 22 obtains a resource allocation request.

[0061] The resource allocation request is used to request to allocate a container for a target task in the computing node 22. In addition, the resource allocation request includes the task content corresponding to the target task (for the sake of distinction, the task content is referred to as task content a hereinafter).

[0062] Exemplarily, when it is needed to allocate resources for the target task, the application manager 212 can send an allocation request to the resource manager 24 according to the process of S301. The allocation request can include the number of containers required by the target task, the container identifier corresponding to the application manager 212, and the like. In addition, the allocation request can also include the task content a corresponding to the target task. Then, the resource manager 24 can send the above-mentioned resource allocation request to the computing node 22 according to the received allocation request.

[0063] The task content a can include, for example, a jar package, an xml file, and other business data required by the target task.

[0064] S402, the computing node 22 preloads the task content a in the container (for example, the container 222), and determines the memory occupied by the container 222.

[0065] The memory size occupied by the container 222 corresponds to the memory occupied by the preloaded task content a.

[0066] Exemplarily, in the computing node 22, the memory can be allocated to the container in a scheduling unit of 100MB. In the process of preloading the task content a in the container 222, the appropriate memory space is allocated to the container 222 according to the requirement, so that the memory size occupied by the container 222 corresponds to the memory occupied by the preloaded task content a.

[0067] S403, the computing node 22 determines to allocate the container 222 for the target task after the preloading of the task content a is completed.

[0068] Specifically, the computing node 22 can refer to the content of S303-S304 after the preloading of the task content a is completed, and can feed back the resource information (which can include the container information of the container 222 and the identifier of the target task, and the like. The resource information is used to indicate that the container 222 is allocated for the target task.) to the resource manager 24, so that the resource manager 24 forwards the resource information to the application manager 212, thereby determining to allocate the container 222 for the target task.

[0069] Therefore, in an implementation mode, the computing node 22 determines to allocate the container 222 for the target task after the preloading of the task content a is completed, which includes that the computing node 22 sends the resource information to the resource manager 24 after the preloading of the task content a is completed.

[0070] The resource information is used to indicate that the container 222 is allocated for the target task.

[0071] In the embodiment of the present application, when the computing node 22 obtains the resource allocation request for requesting to allocate a container for the target task, the computing node 22 can first preload the task content a corresponding to the target task in the container 222 in the computing node 22, and determine the memory occupied by the container 222. The memory size occupied by the container 222 corresponds to the memory occupied by the preloaded task content a. Then, after the task content a is preloaded, the computing node 22 determines to allocate the container 222 for the target task. In this way, after the container 222 is allocated to the target task, the normal running of the target task in the container 222 can be ensured, and the waste of memory resources is avoided, thereby improving the resource use efficiency of the computing node 22.

[0072] In an implementation manner, the method further includes:

[0073] S404, the computing node 22 sets the container 222 to a to-be-pulled state after the task content a is preloaded.

[0074] The to-be-pulled state is used to indicate that the container 222 has been occupied.

[0075] In the implementation manner, by setting the container 222 to the to-be-pulled state after the task content a is preloaded, it can be avoided that the container 222 is occupied by other applications before the application manager issues the target task to the container 222 for running.

[0076] In an implementation manner, the method further includes:

[0077] S405, the computing node 22 receives a running instruction from the application manager 212.

[0078] The running instruction is used to instruct to run the target task in the container 222. For example, the running instruction carries the identifier of the target task and the container 222, so that the computing node 22 determines to run the target task in the container 222 according to the content of the running instruction.

[0079] S406, the computing node 22 runs the target task in the container 222 by using the preloaded task content a.

[0080] In the implementation manner, after the computing node 22 receives the running instruction from the application manager 212, the computing node 22 can run the target task in the container 222 by using the task content a that has been preloaded before. In this way, the time for loading the task content a when the target task is formally run can be saved, and the task running rate is improved.

[0081] For example, Figure 4Fig. 2(a) shows a process diagram of executing a task in each container of a computing node without preloading the task content. As can be seen, each container needs to load the task content into the container through a "loading resource" process before executing the task, and then starts to execute the service. In the case of applying the above implementation manner of the embodiment of the present application, as shown in Fig. 2(b), because the task content has been preloaded in the container before the task execution, after starting to execute, each container can directly start to execute the task without loading the resource again. In this way, the running time of each container can be reduced, and the execution time of the whole application can be reduced. Figure 4

[0082] In addition, in an implementation manner, the embodiment of the present application considers that there can be some task content which is not suitable for preloading.

[0083] Therefore, before executing S402, as shown in Fig. 2(c), the method can further include: Figure 5

[0084] S407, the computing node 22 detects whether the task content a meets a preset condition.

[0085] The preset condition can include that the file format of the task content a meets a preset format. For example, when it is required to limit the installation package file format of the task content to jar format to execute preloading, the above preset format can include jar format. In this way, only when the installation package file format of the task content a is jar format, the computing node 22 preloads the task content a according to the process of S402.

[0086] In addition, the preset condition can further include that the type of the service data of the task content a meets a preset type. For example, when it is required to limit the service data of the task content a to xml type to execute preloading, the above preset type can include xml type. In this way, only when the service data of the task content a is xml type, the computing node 22 preloads the task content a according to the process of S402.

[0087] In addition, the preset condition can further include that the data amount corresponding to the task content a meets a preset data amount. For example, when it is required to limit the data amount of the task content a to be less than 1G to execute preloading, the preset data amount can be 1G. In this way, only when the data amount corresponding to the task content a is less than 1G, the computing node 22 preloads the task content a according to the process of S402.

[0088] Further, S402 can specifically include that the computing node 22 preloads the task content a in the first container after determining that the task content meets the preset condition.

[0089] ​​In addition, as shown in S408, the computing node 22 can assign a container with a preset memory size to the target task, so as to run the target task in the container later. Figure 5

[0090] For example, on one hand, after the task content a is preloaded, the computing node 22 can set the container 222 to the state of being pulled up through S404; on the other hand, in the case that the preloading of the task content a is not successful or the task content a does not meet the preset condition, the computing node 22 can determine to assign a container with a preset memory size to the target task through S408 (i.e., there is no container in the state of being pulled up at this time). Therefore, as shown in S405, the computing node 22 can determine the content of the resource information fed back to the resource manager 24 by judging whether there is a container in the state of being pulled up. Specifically, when it is determined that there is a container in the state of being pulled up, the list of the container in the state of being pulled up is fed back to the resource manager 24, otherwise, the list of the container in the original path (i.e., the list of the container with the preset memory size) is fed back to the resource manager 24. Figure 6

[0091] Further, in a possible design, it is considered that in actual application, the user can set the preset condition used in S407 according to actual needs. Therefore, as shown in S409, before S407, the method can further include: Figure 5

[0092] S409, the computing node 22 receives a first configuration instruction.

[0093] The first configuration instruction is used to determine the preset condition.

[0094] For example, the first configuration instruction can include one or more of the following: the file format of the task content meets a preset format, the type of the service data of the task content meets a preset type, and the data volume corresponding to the task content meets a preset data volume. In this way, the computing node 22 can determine the corresponding preset condition according to the first configuration instruction.

[0095] In addition, as shown in S411, after S409, the method can further include: Figure 5

[0096] S410, the computing node 22 detects whether the preloading configuration is correct.

[0097] For example, after the computing node 22 determines the corresponding preset condition according to the first configuration instruction, the content in the preset condition can be incorrect. Therefore, it can be detected through S410 whether the preloading configuration is correct (including detecting whether the content in the preset condition is correct), so as to ensure the normal operation of the subsequent process.​​​​

[0098] When the computing node 22 detects that the preloading configuration is incorrect, the flow ends. When the computing node 22 detects that the preloading configuration is correct, all or part of the processes in S401-S409 can be performed.

[0099] In another implementation, before S402 is performed, the method can further include: Figure 7

[0100] S411, the computing node 22 determines whether the resource preloading function is enabled.

[0101] In one aspect, when the computing node 22 determines that the resource preloading function is enabled, the computing node 22 preloads the task content a in the first container through the process of S402. Figure 7 In another aspect, as shown in FIG. 4B, when the computing node 22 determines that the resource preloading function is enabled, the computing node 22 first determines whether the task content a meets the preset condition through the content of S407. When it is determined that the task content a meets the preset condition, the computing node 22 preloads the task content a in the first container through the process of S402.

[0102] In another aspect, when the computing node 22 determines that the resource preloading function is disabled, the computing node 22 determines to assign the second container to the target task. The memory size occupied by the second container is a preset value.

[0103] For example, as shown in FIG. 4C, when the computing node 22 determines that the resource preloading function is disabled, the computing node 22 performs S408. Figure 7 In a possible design, considering that in actual application, the resource preloading function can be enabled or disabled according to actual needs of a user. Therefore, as shown in FIG. 4D, before S411, the method can further include:

[0104] Figure 7

[0105] S412, the computing node 22 receives a second configuration instruction.

[0106] The second configuration instruction is used to enable or disable the resource preloading function.

[0107] Based on the method embodiments, the device provided by the embodiments of the present application is described below. The data processing device provided by the embodiments of the present application can be used to implement the functions of the computing node 22 in the foregoing Figures 3-7 .

[0108] For example, as shown in FIG. 5, the data processing device includes a receiving unit 501, a determining unit 502, a preloading unit 503, and a configuring unit 504. Figure 8 ​​​As shown, the data processing apparatus can be a software / hardware module running in the computing node 22, so that when allocating resources for a task, the data processing apparatus can control the computing node 22 to perform all or part of the steps described above. In actual application, the data processing apparatus can be built into the node manager in the computing node 22, thereby serving as a functional module in the node manager in the computing node 22; in addition, the data processing apparatus can also be a functional module independent of the node manager in the computing node 22, as shown. The specific form of the data processing apparatus in the computing node 22 can not be limited in the embodiments of the present application. Figures 3-7 Figure 8 In an implementation manner, as shown in

[0109] In an implementation manner, as shown in Figure 9 is a structural schematic diagram of a data processing apparatus provided by an embodiment of the present application. Specifically, the data processing apparatus 50 can be used for the functions of the computing node described above. Specifically, the data processing apparatus 50 includes:

[0110] The communication unit 501 is configured to obtain a resource allocation request; the resource allocation request is used to request allocation of a container for a target task in the computing node, and the resource allocation request carries task content corresponding to the target task.

[0111] The processing unit 502 is configured to pre-load the task content in the first container and determine the memory occupied by the first container; wherein the memory size occupied by the first container corresponds to the memory occupied by the pre-loaded task content.

[0112] The communication unit 501 is further configured to determine the first container allocated for the target task after the pre-loading of the task content is completed.

[0113] In an implementation manner, the processing unit 502 is further configured to detect whether the task content meets preset conditions; the preset conditions include one or more of the following: a file format of the task content meets a preset format, a type of business data of the task content meets a preset type, and a data volume corresponding to the task content meets a preset data volume.

[0114] The processing unit 502 is configured to pre-load the task content in the first container, including: the processing unit 502 is configured to pre-load the task content in the first container after determining that the task content meets the preset conditions.

[0115] In an implementation manner, the communication unit 501 is further configured to receive a first configuration instruction, the first configuration instruction being used to determine the preset conditions.

[0116] In an implementation manner, the processing unit 502 is configured to pre-load the task content in the first container, including: the processing unit 502 is configured to pre-load the task content in the first container after determining that the resource pre-loading function is enabled.​

[0117] The processing unit 502 is further configured to determine to allocate a second container for the target task after determining that the resource preloading function is closed, and the second container occupies a preset memory size.

[0118] In an implementation manner, the communication unit 501 is further configured to receive a second configuration instruction, and the second configuration instruction is used to start or close the resource preloading function.

[0119] In an implementation manner, the processing unit 502 is further configured to set the first container to a standby state after the task content preloading is completed, and the standby state is used to indicate that the first container has been occupied.

[0120] In an implementation manner, the communication unit 501 is further configured to receive a running instruction from an application manager, and the running instruction is used to indicate to run the target task in the first container.

[0121] The processing unit 502 is further configured to run the target task in the first container by using the preloaded task content.

[0122] In an implementation manner, the computing node is a computing node in a distributed system, and the resource allocation request is sent by a resource manager in the distributed system to the computing node.

[0123] The processing unit 502 is configured to determine to allocate the first container for the target task after the task content preloading is completed, and includes that the processing unit is configured to send resource information to a resource manager after the task content preloading is completed, and the resource information is used to indicate to allocate the first container for the target task.

[0124] Figure 10 Another structure schematic diagram of a data processing apparatus provided in the embodiment is provided. The data processing apparatus 60 can be a chip or a system on chip. The data processing apparatus 60 can include some or all of the following components: a processor 601, a communication line 602, a memory 603, and at least one communication interface 604.

[0125] The processor 601 is configured to execute all or part of the steps performed by the computing node in the embodiment.

[0126] In particular, the processor 601 can include a general-purpose central processing unit (CPU), and can also include a microprocessor, a microcontroller, a field programmable gate array (FPGA), a digital signal processor (DSP), or an application-specific integrated circuit (ASIC), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like.

[0127] In a particular implementation, as one example, the processor 601 can include one or more CPUs, such as CPU0 and CPU1 in Figure 10 .

[0128] In a particular implementation, as one example, the apparatus 60 can include multiple processors, such as the processor 601 and the processor 608 in Figure 10 . Each of these processors can be a single-CPU processor or a multi-CPU processor. A processor here can refer to one or more devices, circuits, and / or processing cores for processing, e.g., computer program instructions.

[0129] In addition, the memory 603 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate synchronous dynamic RAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). The memory 603 can exist independently, and be connected to the processor 601 through the communication line 602. The memory 603 can also be integrated with the processor 601.

[0130] The memory 603 stores computer instructions. The processor 601 can execute the computer instructions stored in the memory 603 to perform all or part of the steps of the method provided in the embodiments.

[0131] Optionally, the computer execution instructions in the embodiments can also be referred to as application program codes, which are not specifically limited in the embodiments.

[0132] In addition, the communication interface 604 uses any transceiver-like device to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0133] In addition, the communication line 602 is used to connect the components in the data processing apparatus 60. Specifically, the communication line 602 can include a data bus, a power supply bus, a control bus, and a state signal bus, etc. However, for the purpose of clear illustration, all kinds of buses are marked as the communication line 602 in the figure.

[0134] In a specific implementation, as an embodiment, the data processing apparatus 60 can further include an output device 607 and an input device 606. The output device 607 can communicate with the processor 601 and can display information in various ways. For example, the output device 607 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device 606 can communicate with the processor 601 and can receive user input in various ways. For example, the input device 606 can be a mouse, a keyboard, a touch screen device, or a sensor device, etc.

[0135] In addition, the data processing apparatus 60 can further include a storage medium 605. The storage medium 605 is used to store computer instructions and various data for implementing the technical solutions of the present embodiment. In order for the data processing apparatus 60 to execute the above-mentioned method of the present embodiment, the computer instructions and various data stored in the storage medium 605 are loaded into the memory 603, so that the processor 601 can execute the computer instructions stored in the memory 603 to execute the method provided by the present embodiment.

[0136] The method steps in the present embodiment can be implemented by hardware or by the processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in a RAM, a flash memory, a ROM, a PROM, an EPROM, an EEPROM, a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in the data processing apparatus. Of course, the processor and the storage medium can also exist as discrete components in the data processing apparatus.

[0137] In the above embodiments, all or part can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments are performed. The computer can be a general purpose computer, a special purpose computer, a computer network, a communication device, a user equipment or other programmable device. The computer programs or instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer programs or instructions can be transferred from one website site, computer, server or data center to another website site, computer, server or data center through wired or wireless manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center and the like integrated with one or more available media. The available media can be a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape; an optical medium, such as a digital video disc (digital video disc, DVD); and a semiconductor medium, such as an SSD.

[0138] In the embodiments, the terms and / or descriptions among different implementation manners are consistent and can be referred to each other if there is no special description and logical conflict, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0139] In the embodiments, "at least one" means one or more, "multiple" means two or more, and other quantifiers are similar. The association relationship of the associated objects is described by "and / or", which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, for the elements (element) appearing in the singular form "a", "an" and "the", unless the context clearly specifies otherwise, it does not mean "one or only one", but means "one or more than one". For example, "a device" means one or more such devices. Furthermore, "at least one of" means one or any combination of the subsequent associated objects, for example, "at least one of A, B and C" includes A, B, C, AB, AC, BC or ABC. In the literal description of the embodiments, the character " / ", generally represents that the preceding and following associated objects are in an "or" relationship; in the formulas of the embodiments, the character " / ", represents that the preceding and following associated objects are in a "division" relationship.

Claims

1. A resource scheduling method, characterized in that, The method comprises: The computing node acquires a resource allocation request; the resource allocation request is used to request allocation of a container for a target task in the computing node, and the target task content is included in the resource allocation request; The computing node preloads the task content in the first container and determines the memory occupied by the first container; wherein the memory size occupied by the first container corresponds to the memory occupied by the preloaded task content; The computing node determines to allocate the first container for the target task after the task content is preloaded.

2. The method of claim 1, wherein, The method further comprises: The computing node detects whether the task content meets the preset condition; The preset condition comprises one or more of the following: the file format of the task content meets the preset format, the type of the service data of the task content meets the preset type, and the data volume corresponding to the task content meets the preset data volume; The computing node preloads the task content in the first container, comprising: the computing node preloads the task content in the first container after determining that the task content meets the preset condition.

3. The method of claim 2, wherein, The method further comprises: The computing node receives a first configuration instruction, and the first configuration instruction is used to determine the preset condition.

4. The method according to any one of claims 1 to 3, characterized in that, The computing node preloads the task content in the first container, comprising: the computing node preloads the task content in the first container after determining that the resource preloading function is enabled; The method further comprises: The computing node determines to allocate a second container for the target task after determining that the resource preloading function is disabled, and the memory size occupied by the second container is a preset value.

5. The method of claim 4, wherein, The method further comprises: the computing node receives a second configuration instruction, and the second configuration instruction is used to enable or disable the resource preloading function.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: after the task content is preloaded, the computing node sets the first container to a standby state, and the standby state is used to indicate that the first container has been occupied.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: The computing node receives a running instruction from an application manager, and the running instruction is used to instruct to run the target task in the first container; The computing node runs the target task in the first container by using the preloaded task content.

8. The method according to any one of claims 1 to 7, characterized in that, The computing node is a computing node in a distributed system, and the resource allocation request is sent by a resource manager in the distributed system to the computing node; The computing node determines to allocate the first container for the target task after the task content is preloaded, comprising: the computing node sends resource information to the resource manager after the task content is preloaded, and the resource information is used to indicate that the first container is allocated for the target task.

9. A data processing apparatus, characterized by, The data processing device is applied to a computing node; the data processing device comprises: A communication unit is configured to acquire a resource allocation request; the resource allocation request is used to request allocation of a container for a target task in the computing node, and the target task content is included in the resource allocation request; a processing unit, configured to preload the task content in a first container, and determine memory occupied by the first container; wherein the memory occupied by the first container corresponds to memory occupied by the preloaded task content; the communication unit is further configured to, after the preloading of the task content is completed, allocate the first container to the target task.

10. A data processing apparatus, characterized by, The computer program product comprises instructions which, when executed on a processor, implement the method according to any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores instructions, and when the instructions are executed on a processor, the method according to any one of claims 1-8 is implemented.

12. A computer program product, characterised in that, The computer program product comprises instructions which, when executed on a processor, implement the method according to any one of claims 1-8.

Images