A resource allocation method and device

Abstract

The present disclosure provides a method and device for resource allocation, which first receives resource information of multiple nodes in a cluster through an application manager in the cluster; secondly, after receiving resource demand information for a target application, based on the resource information of the multiple nodes, determines the target resource information of the target node that matches the resource demand information; the resource demand information carries resource configuration information of at least one container and demand information of the node in the cluster where the resource to be configured is located; finally, based on the determined target resource information of the target node, sends a resource allocation request to the resource manager of the cluster. The present disclosure enables the resources allocated to the application to meet the application's demand for the node, thereby improving the compatibility between the application and the allocated node, and reducing the situation where the application cannot run normally.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a resource allocation method and device. Background Art

[0002] At present, distributed systems have been widely used. A distributed system is a system composed of a group of computer nodes that communicate through a network and coordinate work to complete a common task. A distributed system can use cheap, ordinary machines to complete computing and storage tasks that a single computer cannot complete. Hadoop is a distributed system infrastructure, and Yet Another Resource Negotiator (YARN) is a Hadoop resource manager. When an application needs to allocate resources, YARN can select appropriate resources from the nodes in the cluster for the application, and the resources are allocated in the form of containers.

[0003] However, when allocating resources, the nodes and resources allocated to the application are determined by YARN, and YARN cannot consider the application's requirements for the nodes in the cluster when allocating resources, resulting in poor compatibility between the application and the container of the allocated node, affecting the normal operation of the application. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a resource allocation method and device, which can enable the resources allocated to the application to meet the application's requirements for nodes, thereby improving the compatibility between the application and the allocated nodes and reducing the situation where the application cannot run normally.

[0005] In a first aspect, an embodiment of the present disclosure provides a resource allocation method, the method comprising:

[0006] Receive resource information of multiple nodes in the cluster, where the resource information of the nodes includes attribute information of the nodes and container information of the nodes;

[0007] Receive resource requirement information for a target application sent by a client; the resource requirement information includes resource configuration information of at least one container and requirement information of a node in the cluster where the resource to be configured is located;

[0008] Based on the resource information of the multiple nodes, determine target resource information of a target node matching the resource demand information;

[0009] Based on the determined target resource information of the target node, a resource allocation request is sent to a resource manager of the cluster.

[0010] In a possible implementation, receiving resource information of multiple nodes in a cluster includes:

[0011] Sending a node resource information query request to the resource manager;

[0012] The resource information respectively corresponding to the plurality of nodes fed back by the resource manager is received, and after receiving the updated resource information of at least one node fed back by the resource manager, the stored resource information of the at least one node is updated.

[0013] In a possible implementation manner, the container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information;

[0014] The demand information for the nodes in the cluster where the resources to be configured are located includes: at least one of the demand for the resource type of the node, the restriction conditions for the running container, the restriction conditions for the address information of the node, the restriction conditions for the operation status information of the node and the demand for idle resources.

[0015] In a possible implementation, determining, based on the resource information of the multiple nodes, target resource information of a target node matching the resource requirement information includes:

[0016] Based on the resource information of the multiple nodes, determining at least one candidate node matching the resource requirement information;

[0017] The at least one candidate node and the resource information of the candidate node are sent to the client, a target node selected from the candidate nodes by a user according to the resource information of the candidate node and sent by the client is received, and target resource information of the target node is determined according to the resource demand information.

[0018] In a possible implementation, the method further includes:

[0019] In response to the resource release information for the target application, a resource release request is sent to the resource manager so that the resource manager releases the target resource information of the target node.

[0020] In a possible implementation, the method further includes:

[0021] If resource allocation failure information is received from the resource manager, a resource allocation failure prompt is sent to the client, and the step of determining the target resource information of the target node that matches the resource information based on the resource information is re-executed, and the step of sending a resource allocation request to the resource manager of the cluster based on the determined target resource information of the target node is re-executed.

[0022] In a second aspect, the embodiment of the present disclosure also provides another resource allocation method, the method comprising:

[0023] Receive resource information of multiple nodes in the cluster; the resource information of the node includes attribute information of the node and container information of the node;

[0024] In response to a resource allocation instruction for a target application, the resource information of the plurality of nodes is sent to a client;

[0025] receiving resource allocation information for the target application sent by the client, wherein the resource allocation information is determined based on resource requirement information of the target application and the resource information of the multiple nodes, and is used to instruct a target node selected from the multiple nodes to execute the target application;

[0026] Based on the resource allocation information of the target node, a resource allocation request is sent to a resource manager of a cluster.

[0027] In a third aspect, the present disclosure also provides another resource allocation method, the method comprising:

[0028] Monitor resource information of multiple nodes in the cluster;

[0029] In response to preset trigger conditions, the resource information of multiple nodes in the cluster is sent to the application manager;

[0030] In response to the resource allocation request sent by the application manager, target resource information indicating a target node in the resource allocation request is allocated to a target application program indicated by the resource allocation request.

[0031] In a possible implementation manner, the method further includes:

[0032] If the target resource information has been occupied by other applications except the target application, resource allocation failure information is sent to the application manager.

[0033] In a fourth aspect, the embodiment of the present disclosure also provides another resource allocation method, including:

[0034] Acquire resource information of multiple nodes and display the resource information in a preset format; the resource information of the nodes includes attribute information and container information of the nodes;

[0035] receiving resource allocation information for a target application submitted by a user, wherein the resource allocation information is determined based on resource requirement information of the target application and the resource information of the plurality of nodes, and is used to indicate a target node selected from the plurality of nodes to execute the target application;

[0036] The resource allocation information is sent to the application manager.

[0037] In a fifth aspect, an embodiment of the present disclosure further provides a resource allocation device, the device comprising:

[0038] A first receiving module is used to receive resource information of multiple nodes in the cluster, where the resource information of the nodes includes attribute information of the nodes and container information of the nodes;

[0039] A second receiving module is used to receive resource demand information for a target application sent by a client; the resource demand information includes resource configuration information of at least one container and demand information of a node in the cluster where the resource to be configured is located;

[0040] A first determining module, configured to determine target resource information of a target node matching the resource demand information based on the resource information of the multiple nodes;

[0041] The first sending module is used to send a resource allocation request to a resource manager of the cluster based on the determined target resource information of the target node.

[0042] In a possible implementation manner, the first receiving module is specifically configured to:

[0043] Sending a node resource information query request to the resource manager;

[0044] The resource information respectively corresponding to the plurality of nodes fed back by the resource manager is received, and after receiving the updated resource information of at least one node fed back by the resource manager, the stored resource information of the at least one node is updated.

[0045] In a possible implementation manner, the container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information;

[0046] The demand information for the nodes in the cluster where the resources to be configured are located includes: at least one of the demand for the resource type of the node, the restriction conditions for the running container, the restriction conditions for the address information of the node, the restriction conditions for the operation status information of the node and the demand for idle resources.

[0047] In a possible implementation manner, the first determining module is specifically configured to:

[0048] Based on the resource information of the multiple nodes, determining at least one candidate node matching the resource requirement information;

[0049] The at least one candidate node and the resource information of the candidate node are sent to the client, a target node selected from the candidate nodes by a user according to the resource information of the candidate node and sent by the client is received, and target resource information of the target node is determined according to the resource demand information.

[0050] In a possible implementation manner, the first sending module is further configured to:

[0051] In response to the resource release information for the target application, a resource release request is sent to the resource manager so that the resource manager releases the target resource information of the target node.

[0052] In a possible implementation manner, the first sending module is further configured to:

[0053] If resource allocation failure information is received from the resource manager, a resource allocation failure prompt is sent to the client, and the resource allocation device is caused to re-execute the step of determining the target resource information of the target node that matches the resource information based on the resource information, and the step of sending a resource allocation request to the resource manager of the cluster based on the determined target resource information of the target node.

[0054] In a sixth aspect, the embodiment of the present disclosure also provides another resource allocation device, the device comprising:

[0055] A third receiving module is used to receive resource information of multiple nodes in the cluster; the resource information of the node includes attribute information of the node and container information of the node;

[0056] A second sending module, configured to respond to a resource allocation instruction for a target application and send resource information of the plurality of nodes to a client;

[0057] a fourth receiving module, configured to receive resource allocation information for the target application sent by the client, wherein the resource allocation information is determined based on the resource requirement information of the target application and the resource information of the plurality of nodes, and is configured to instruct a target node selected from the plurality of nodes to execute the target application;

[0058] The third sending module is used to send a resource allocation request to a resource manager of the cluster based on the resource allocation information of the target node.

[0059] In a seventh aspect, the embodiments of the present disclosure further provide another resource allocation device, the device comprising:

[0060] Monitoring module, used to monitor resource information of multiple nodes in the cluster;

[0061] A fourth sending module, configured to respond to a preset trigger condition and send resource information of multiple nodes in the cluster to the application manager;

[0062] The allocation module is used to respond to the resource allocation request sent by the application manager and allocate the target resource information indicating the target node in the resource allocation request to the target application indicated by the resource allocation request.

[0063] In a possible implementation manner, the fourth sending module is further used for:

[0064] If the target resource information has been occupied by other applications except the target application, resource allocation failure information is sent to the application manager.

[0065] In an eighth aspect, the embodiment of the present disclosure also provides another resource allocation device, the device comprising:

[0066] An acquisition module, used to acquire resource information of multiple nodes and display the resource information in a preset format; the resource information of the node includes attribute information and container information of the node;

[0067] a fifth receiving module, configured to receive resource allocation information for a target application submitted by a user, wherein the resource allocation information is determined based on resource requirement information of the target application and resource information of the plurality of nodes, and is configured to instruct a target node selected from the plurality of nodes to execute the target application;

[0068] The fifth sending module is used to send the resource allocation information to the application manager.

[0069] In the ninth aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, steps in any possible implementation of the above-mentioned resource allocation method are performed.

[0070] In a tenth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in any possible implementation of the above-mentioned resource allocation method are executed.

[0071] For a description of the effects of the above-mentioned resource allocation device, electronic device, and computer-readable storage medium, please refer to the description of the above-mentioned resource allocation method, which will not be repeated here.

[0072] The resource allocation method and device provided by the embodiment of the present disclosure first receives resource information of multiple nodes in the cluster through the application manager in the cluster; secondly, after receiving the resource demand information for the target application, based on the resource information of the multiple nodes, the target resource information of the target node matching the resource demand information is determined; the resource demand information carries the resource configuration information of at least one container and the demand information of the node in the cluster where the resource to be configured is located; finally, based on the determined target resource information of the target node, a resource allocation request is sent to the resource manager of the cluster. The present disclosure can allocate resources for the application according to the resource configuration information of at least one container carried in the resource demand information and the demand information of the node in the cluster where the resource to be configured is located, so that the resources allocated to the application can meet the application's demand for the node, thereby improving the compatibility between the application and the allocated node, and reducing the situation where the application cannot run normally.

[0073] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0075] Figure 1 A schematic diagram of a resource allocation system provided by an embodiment of the present disclosure is shown;

[0076] Figure 2 A flow chart of a resource allocation method provided by an embodiment of the present disclosure is shown;

[0077] Figure 3 A flowchart of another resource allocation method provided by an embodiment of the present disclosure is shown;

[0078] Figure 4 A flow chart of another resource allocation method provided by an embodiment of the present disclosure is shown;

[0079] Figure 5 A flowchart of another resource allocation method provided by an embodiment of the present disclosure is shown;

[0080] Figure 6 A schematic diagram of a resource allocation device provided by an embodiment of the present disclosure is shown;

[0081] Figure 7 A schematic diagram showing another resource allocation device provided by an embodiment of the present disclosure;

[0082] Figure 8 A schematic diagram showing another resource allocation device provided by an embodiment of the present disclosure is shown;

[0083] Fig. 9 A schematic diagram showing another resource allocation device provided by an embodiment of the present disclosure;

[0084] Fig.10 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0086] Research has found that when a distributed system is allocating resources, YARN can schedule containers that match the resource configuration information carried in the resource scheduling request to nodes. In this resource allocation method, the container to which the container is scheduled is determined by YARN. Since YARN only considers the resource configuration information required by the container when allocating resources, the node allocated to the container only meets the container's resource requirements, without considering the container's demand information for the node itself, which can easily lead to poor compatibility between the node allocated to the container and the container itself, and thus cause the application to fail to run normally.

[0087] Based on the above research, the present disclosure provides a resource allocation method and device, so that the container allocated to the application meets the requirements for the node and the container itself, thereby improving the compatibility between the application and the container allocated to the node, and reducing the situation where the application cannot run normally.

[0088] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0089] The technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0090] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0091] In order to facilitate the use of this embodiment, a resource allocation system disclosed in the embodiment of the present disclosure is first introduced. Figure 1As shown, it is a schematic diagram of the resource allocation system provided by the embodiment of the present disclosure. The resource allocation system provided by the embodiment of the present disclosure can be a distributed system, which can operate in the form of a cluster. In the cluster, it can include a client, multiple nodes, a node manager (Node Manager, NM) corresponding to each distributed node, a resource manager (Resource Manager, RM) and an application manager (Application Master, AM), wherein the client can communicate with the application manager AM, and the node can be a computer device with a certain computing power; the application manager AM manages each instance of an application running in YARN, is responsible for coordinating resources from RM, and monitors the execution and resource usage of the container (resource allocation of CPU, memory, etc.) through NM; the node manager NM manages each node in a YARN cluster, and provides services for each node in the cluster, from supervising the lifelong management of a container to monitoring resources and tracking node health; the application manager AM controls the entire cluster and manages the allocation of applications to basic computing resources, and also allocates resources with AM, and starts and monitors their basic applications with NM.

[0092] To facilitate understanding of the present embodiment, a resource allocation method disclosed in the embodiment of the present disclosure is introduced in detail. The resource allocation method provided in the embodiment of the present disclosure is described below by taking the execution subject as an application manager AM in a cluster as an example.

[0093] See also Figure 2 As shown, it is a flow chart of a resource allocation method provided by an embodiment of the present disclosure, and its execution subject may be an application manager AM in a cluster, including steps S201 to S203, wherein:

[0094] S201. Receive resource information of multiple nodes in a cluster, where the resource information of the nodes includes attribute information of the nodes and container information of the nodes.

[0095] In this step, the resource manager RM in the YARN cluster can periodically obtain the resource information of the nodes from the node managers NM corresponding to multiple nodes in the cluster, and send the resource information to the application manager AM after receiving the resource information. After obtaining the resource information, the application manager AM can store the received resource information. When the application manager AM receives the resource information again, it can update the previously received resource information to keep the stored resource information up to date.

[0096] The resource information of a node may include the attribute information of the node and the attribute information of the container allocated on the node, i.e., the container information of the node. The attribute information of a node may include the total amount of resources on the node, such as the total amount of central processing unit (CPU) on the node, the total amount of memory, the density of the CPU, whether it contains a graphics processing unit (GPU), the type information of the GPU, and the operation status of the node, such as the load status and health status of the node, and the number of containers on the node; the container information of a node may include the resource occupancy of the container, such as the number of CPUs and memory size occupied by the container, as well as the type information of the container, such as the hbase type container, and other information of the container.

[0097] Exemplarily, resource information can be stored in a tree-like directory, in which a directory for each node is set in the first level of the tree-like directory, a directory for attribute information corresponding to the node is set under the directory of each node, and a directory for containers deployed on the node is set, and the attribute information of each container is stored under the directory of the containers deployed on the node.

[0098] S202, receiving resource requirement information for a target application sent by a client; the resource requirement information includes resource configuration information of at least one container and requirement information for nodes in the cluster where resources to be configured are located.

[0099] In this step, the application manager AM may receive resource requirement information for the target application sent by the client, and the resource requirement information may carry resource configuration information of at least one container and requirement information for the node.

[0100] The resource configuration information may include the number of containers required by the target application, the number of CPUs that the container should occupy, the memory size, and the container type. Resource configuration information is the information that YARN needs to allocate containers for applications. The node demand information may include the number of CPUs, total memory, whether a GPU is required, the health of the node, the load, the type of containers already on the node, and so on.

[0101] S203: Based on the resource information of the multiple nodes, determine target resource information of a target node that matches the resource requirement information.

[0102] After receiving the resource requirement information for the target program, the application manager AM can filter out at least one node that meets the resource requirement information from multiple nodes based on the resource information of multiple nodes stored. The filtered node should simultaneously meet the resource configuration information for the container in the resource requirement information and the requirement information for the nodes in the cluster.

[0103] After screening out at least one node, the application manager AM can select another node from the at least one screened node as the target node. The selection method can be random selection or selection according to a preset priority. For example, the node with the largest available memory is preferentially selected as the target node.

[0104] After determining the target node, the target resource information to be allocated to the target application can be determined according to the resource configuration information in the resource demand information. The target resource information may include identification information of at least one CPU in the target node, memory size, etc.

[0105] S204: Send a resource allocation request to a resource manager of the cluster based on the determined target resource information of the target node.

[0106] After determining the target resource information of the target node, a resource allocation request corresponding to the target resource information can be generated. The resource allocation request indicates the target node and the resources in the target node that need to be allocated to the target application, that is, the resource allocation request indicates information about the container allocated to the target resource.

[0107] After receiving the resource allocation request, the resource manager RM may set a container on the target node according to the content indicated in the resource allocation request, wherein the container includes the resources indicated in the resource allocation request.

[0108] After the resource manager RM allocates the container on the target node, the application manager AM can request the resource manager RM to start or stop the container through a preset protocol, and the resource manager RM can feed back the corresponding start result or stop result to the application manager.

[0109] The resource allocation method provided by the embodiment of the present disclosure first receives resource information of multiple nodes in the cluster through the application manager in the cluster; secondly, after receiving the resource demand information for the target application, based on the resource information of the multiple nodes, the target resource information of the target node matching the resource demand information is determined; the resource demand information carries the resource configuration information of at least one container and the demand information of the node in the cluster where the resource to be configured is located; finally, based on the determined target resource information of the target node, a resource allocation request is sent to the resource manager of the cluster. The present disclosure can allocate resources for the application according to the resource configuration information of at least one container carried in the resource demand information and the demand information of the node in the cluster where the resource to be configured is located, so that the resources allocated to the application can meet the application's demand for the node, thereby improving the compatibility between the application and the allocated node, and reducing the situation where the application cannot run normally.

[0110] In a possible implementation, receiving resource information of multiple nodes in a cluster includes:

[0111] Sending a node resource information query request to the resource manager;

[0112] The resource information respectively corresponding to the plurality of nodes fed back by the resource manager is received, and after receiving the updated resource information of at least one node fed back by the resource manager, the stored resource information of the at least one node is updated.

[0113] In this step, the application manager AM can actively send a node resource information query request to the resource manager RM. After receiving the node resource information query request, the resource manager RM can request the node resource information from the node manager NM, and then send the resource information fed back by the node manager NM to the application manager AM. It can also directly send its own stored resource information to the application manager AM. After the resource manager RM feeds back the resource information, whenever the resource manager RM obtains updated resource information, it sends the updated resource information to the application manager AM. After the application manager AM receives the updated resource information of the node fed back by the resource manager RM, it updates the stored node resource information according to the updated resource information.

[0114] In a possible implementation manner, the container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information;

[0115] The demand information for the nodes in the cluster where the resources to be configured are located includes: at least one of the demand for the resource type of the node, the restriction conditions for the running container, the restriction conditions for the address information of the node, the restriction conditions for the operation status information of the node and the demand for idle resources.

[0116] The resource information may include at least one of the above information, and the demand information may be changed accordingly according to the type of information included in the resource information.

[0117] Specifically, the information type in the demand information should be included in the resource information. For example, if the resource information includes the resource type of the node and the type of the running container, the demand information may include the demand for the resource type of the node and / or the restriction conditions on the running container.

[0118] In a possible implementation, determining, based on the resource information of the multiple nodes, target resource information of a target node matching the resource requirement information includes:

[0119] Based on the resource information of the multiple nodes, determining at least one candidate node matching the resource requirement information;

[0120] The at least one candidate node and the resource information of the candidate node are sent to the client, a target node selected from the candidate nodes by a user according to the resource information of the candidate node and sent by the client is received, and target resource information of the target node is determined according to the resource demand information.

[0121] In this step, the application manager AM can first screen out at least one candidate node that meets the resource demand information from multiple nodes based on the resource information, and then notify the user of the candidate nodes and the resource information of the candidate nodes, so that the user can select the target node from at least one candidate node according to his or her own needs.

[0122] After obtaining the target node selected by the user, target resource information of the resources to be allocated to the target application can be determined based on the resource demand information.

[0123] In a possible implementation, the method further includes:

[0124] In response to the resource release information for the target application, a resource release request is sent to the resource manager so that the resource manager releases the target resource information of the target node.

[0125] In this step, the user may no longer need to use the target application. At this time, the user can send resource release information to the application manager AM through the application. After receiving the resource release information, the application manager AM can generate a resource release request and send it to the resource manager RM, so that the resource manager can release the resources corresponding to the target resource information.

[0126] In a possible implementation, the method further includes:

[0127] If resource allocation failure information is received from the resource manager, a resource allocation failure prompt is sent to the client, and the step of determining the target resource information of the target node that matches the resource information based on the resource information is re-executed, and the step of sending a resource allocation request to the resource manager of the cluster based on the determined target resource information of the target node is re-executed.

[0128] In this step, the resources requested from the resource manager RM may have been occupied. If RM fails to allocate resources, the resource allocation failure information can be returned to AM. After receiving the resource allocation failure information, AM can send a resource allocation failure prompt to the client, and can also execute the step of determining the target resource information again and send a new resource allocation request to RM.

[0129] See also Figure 3 As shown, it is a flowchart of another resource allocation method provided by an embodiment of the present disclosure, and the execution subject may be an application manager AM in a cluster, including steps S301 to S304, wherein:

[0130] S301. Receive resource information of multiple nodes in a cluster; the resource information of the nodes includes attribute information of the nodes and container information of the nodes.

[0131] S302: In response to a resource allocation instruction for a target application, the resource information of the plurality of nodes is sent to a client.

[0132] The resource allocation trigger instruction may be triggered by a user through a client of a target application. When the application manager AM detects the resource allocation trigger instruction, it may provide the acquired resource information of multiple nodes to the user.

[0133] Specifically, resource information of multiple nodes may be displayed to a user through a client of a target application.

[0134] S303, receiving resource allocation information for the target application sent by the client, wherein the resource allocation information is determined based on resource requirement information of the target application and the resource information of the multiple nodes, and is used to indicate a target node selected from the multiple nodes to execute the target application.

[0135] In this step, after the user obtains the resource information of multiple nodes, the user can select the target node from the multiple nodes according to the resource requirement information of the target application and the resource information of the multiple nodes, and determine the resource allocation information. Here, the resource allocation information can be information that directly indicates the target resource information of the target node.

[0136] S304: Send a resource allocation request to a resource manager of a cluster based on the resource allocation information of the target node.

[0137] Among them, the explanation and principle of some contents in steps S301 to S304 are the same as those in steps S201 to S204, and will not be repeated here.

[0138] See also Figure 4As shown, it is a flowchart of another resource allocation method provided by an embodiment of the present disclosure, and the execution subject may be a resource manager RM in a cluster, including steps S401 to S403, wherein:

[0139] S401. Monitor resource information of multiple nodes in a cluster.

[0140] In this step, the resource manager RM can monitor the resource information of multiple nodes in the cluster through the node manager NM.

[0141] Specifically, the resource manager RM may periodically send a resource information acquisition request to the node manager NM, and receive the resource information of the node fed back by the node manager NM.

[0142] S402: In response to a preset trigger condition, send resource information of multiple nodes in the cluster to an application manager.

[0143] The preset triggering condition may include receiving a resource information acquisition request from the application manager AM, or a preset time has passed after sending resource information to the application manager AM, or receiving updated resource information.

[0144] S403: In response to the resource allocation request sent by the application manager, allocate the target resource information of the target node indicated in the resource allocation request to the target application indicated by the resource allocation request.

[0145] In this step, after the resource allocation request sent by the application manager AM is detected, the target resource allocation information of the target node indicated in the resource allocation request may be allocated to the target application so that the target application obtains the resources required for operation.

[0146] In a possible implementation manner, if the target resource information has been occupied by other applications except the target application, resource allocation failure information is sent to the application manager so that the application manager reallocates resources.

[0147] See also Figure 5 As shown, it is a flowchart of another resource allocation method provided by an embodiment of the present disclosure, and the execution subject may be a client, including steps S501 to S503, wherein:

[0148] S501: Acquire resource information of multiple nodes, and display the resource information in a preset format; the resource information of the nodes includes attribute information and container information of the nodes.

[0149] In this step, the resource information of the node can be obtained from the application manager AM in the cluster. After obtaining the resource information of multiple nodes, the resource information can be displayed in a preset format so that the user can view the resource information and determine the resource allocation information based on the resource information and the resource demand information of the target application.

[0150] S502: Receive resource allocation information for a target application submitted by a user, wherein the resource allocation information is determined based on resource requirement information of the target application and the resource information of the multiple nodes, and is used to indicate a target node selected from the multiple nodes to execute the target application.

[0151] In this step, the client may receive resource allocation information submitted by the user. The resource allocation information may include a target node selected by the user from multiple nodes and resource demand information of a target application.

[0152] S503: Send the resource allocation information to the application manager.

[0153] In this step, the client can send the resource allocation information to the AM, and the AM can send the resource allocation information to the resource manager RM, so that the RM allocates resources to the target application according to the resource allocation information, so that the target application is executed by the target node.

[0154] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0155] Based on the same inventive concept, a resource allocation device corresponding to the resource allocation method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned resource allocation method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0156] Reference Figure 6 FIG. 6 is a schematic diagram of a resource allocation device provided in an embodiment of the present disclosure, wherein the resource allocation device 600 includes:

[0157] A first receiving module 610 is used to receive resource information of multiple nodes in the cluster, where the resource information of the nodes includes attribute information of the nodes and container information of the nodes;

[0158] The second receiving module 620 is used to receive resource demand information for a target application sent by a client; the resource demand information includes resource configuration information of at least one container and demand information of a node in the cluster where the resource to be configured is located;

[0159] A first determination module 630, configured to determine target resource information of a target node matching the resource requirement information based on the resource information of the multiple nodes;

[0160] The first sending module 640 is configured to send a resource allocation request to a resource manager of the cluster based on the determined target resource information of the target node.

[0161] In a possible implementation manner, the first receiving module 610 is specifically configured to:

[0162] Sending a node resource information query request to the resource manager;

[0163] The resource information respectively corresponding to the plurality of nodes fed back by the resource manager is received, and after receiving the updated resource information of at least one node fed back by the resource manager, the stored resource information of the at least one node is updated.

[0164] In a possible implementation manner, the container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information;

[0165] The demand information for the nodes in the cluster where the resources to be configured are located includes: at least one of the demand for the resource type of the node, the restriction conditions for the running container, the restriction conditions for the address information of the node, the restriction conditions for the operation status information of the node and the demand for idle resources.

[0166] In a possible implementation manner, the first determining module 630 is specifically configured to:

[0167] Based on the resource information of the multiple nodes, determining at least one candidate node matching the resource requirement information;

[0168] The at least one candidate node and the resource information of the candidate node are sent to the client, a target node selected from the candidate nodes by a user according to the resource information of the candidate node and sent by the client is received, and target resource information of the target node is determined according to the resource demand information.

[0169] In a possible implementation manner, the first sending module 640 is further configured to:

[0170] In response to the resource release information for the target application, a resource release request is sent to the resource manager so that the resource manager releases the target resource information of the target node.

[0171] In a possible implementation manner, the first sending module 640 is further configured to:

[0172] If resource allocation failure information is received from the resource manager, a resource allocation failure prompt is sent to the client, and the resource allocation device is caused to re-execute the step of determining the target resource information of the target node that matches the resource information based on the resource information, and the step of sending a resource allocation request to the resource manager of the cluster based on the determined target resource information of the target node.

[0173] Reference Figure 7 FIG. 1 is a schematic diagram of another resource allocation device provided in an embodiment of the present disclosure, wherein the resource allocation device 700 includes:

[0174] The third receiving module 710 is used to receive resource information of multiple nodes in the cluster; the resource information of the node includes attribute information of the node and container information of the node;

[0175] A second sending module 720, configured to respond to a resource allocation instruction for a target application and send the resource information of the plurality of nodes to a client;

[0176] A fourth receiving module 730 is used to receive resource allocation information for the target application sent by the client, wherein the resource allocation information is determined based on the resource requirement information of the target application and the resource information of the multiple nodes, and is used to instruct a target node selected from the multiple nodes to execute the target application;

[0177] The third sending module 740 is configured to send a resource allocation request to a resource manager of a cluster based on the resource allocation information of the target node.

[0178] Reference Figure 8 FIG. 8 is a schematic diagram of another resource allocation device provided in an embodiment of the present disclosure. The resource allocation device 800 includes:

[0179] Monitoring module 810, used to monitor resource information of multiple nodes in the cluster;

[0180] A fourth sending module 820, configured to send resource information of multiple nodes in the cluster to the application manager in response to a preset trigger condition;

[0181] The allocation module 830 is used to respond to the resource allocation request sent by the application manager and allocate the target resource information indicating the target node in the resource allocation request to the target application indicated by the resource allocation request.

[0182] In a possible implementation manner, the fourth sending module 820 is further configured to:

[0183] If the target resource information has been occupied by other applications except the target application, resource allocation failure information is sent to the application manager.

[0184] Reference Fig. 9 FIG. 1 is a schematic diagram of another resource allocation device provided in an embodiment of the present disclosure, wherein the resource allocation device 900 includes:

[0185] The acquisition module 910 is used to acquire resource information of multiple nodes and display the resource information in a preset format; the resource information of the node includes attribute information and container information of the node;

[0186] A fifth receiving module 920, configured to receive resource allocation information for a target application submitted by a user, wherein the resource allocation information is determined based on resource requirement information of the target application and the resource information of the plurality of nodes, and is configured to indicate a target node selected from the plurality of nodes to execute the target application;

[0187] The fifth sending module 930 is configured to send the resource allocation information to the application manager.

[0188] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0189] Based on the same technical concept, the embodiment of the present application also provides an electronic device. Fig.10 As shown, it is a structural diagram of an electronic device 1000 provided in an embodiment of the present application, including a processor 1001, a memory 1002, and a bus 1003. Among them, the memory 1002 is used to store execution instructions, including a memory 10021 and an external memory 10022; the memory 10021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 1001, and the data exchanged with the external memory 10022 such as a hard disk. The processor 1001 exchanges data with the external memory 10022 through the memory 10021. When the electronic device 1000 is running, the processor 1001 communicates with the memory 1002 through the bus 1003, so that the processor 1001 executes the steps in the above-mentioned resource allocation method. For details, please refer to the above-mentioned method embodiment, which will not be repeated here.

[0190] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the resource allocation method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0191] The computer program product of the resource allocation method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the resource allocation method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0192] The present disclosure also provides a computer program, which implements any one of the methods of the aforementioned embodiments when executed by a processor. The computer program product can be implemented in hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0193] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0196] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0197] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A resource allocation method, characterized in that: The method comprises: Receive resource information of multiple nodes in the cluster, where the resource information of the nodes includes attribute information of the nodes and container information of the nodes, wherein the container information of the nodes is attribute information of the containers running on the nodes; Receive resource requirement information for a target application sent by a client; the resource requirement information includes resource configuration information of at least one container and requirement information of a node in the cluster where the resource to be configured is located; Based on the resource information of the multiple nodes, determine target resource information of a target node matching the resource demand information; Sending a resource allocation request to a resource manager of the cluster based on the determined target resource information of the target node; The container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information; Among them, the demand information for the nodes in the cluster where the resources to be configured are located includes: requirements for the resource type of the node, restrictions on the running container, restrictions on the address information of the node, restrictions on the operating status information of the node, and at least one of the requirements for the idle resources.

2. The method according to claim 1, characterized in that: Receive resource information of multiple nodes in the cluster, including: Sending a node resource information query request to the resource manager; The resource information respectively corresponding to the plurality of nodes fed back by the resource manager is received, and after receiving the updated resource information of at least one node fed back by the resource manager, the stored resource information of the at least one node is updated.

3. The method according to claim 1, characterized in that Determining target resource information of a target node matching the resource requirement information based on the resource information of the multiple nodes includes: Based on the resource information of the multiple nodes, determining at least one candidate node matching the resource requirement information; The at least one candidate node and the resource information of the candidate node are sent to the client, a target node selected from the candidate nodes by a user according to the resource information of the candidate node and sent by the client is received, and target resource information of the target node is determined according to the resource demand information.

4. The method according to claim 1, characterized in that: Also includes: In response to the resource release information for the target application, a resource release request is sent to the resource manager so that the resource manager releases the target resource information of the target node.

5. The method according to claim 1, characterized in that The method further comprises: If resource allocation failure information is received from the resource manager, a resource allocation failure prompt is sent to the client, and the step of determining the target resource information of the target node that matches the resource information based on the resource information is re-executed, and the step of sending a resource allocation request to the resource manager of the cluster based on the determined target resource information of the target node is re-executed.

6. A resource allocation method, characterized in that: The method comprises: Receive resource information of multiple nodes in the cluster; the resource information of the node includes attribute information of the node and container information of the node, wherein the container information of the node is attribute information of the container running on the node; In response to a resource allocation instruction for a target application, the resource information of the plurality of nodes is sent to a client; receiving resource allocation information for the target application sent by the client, wherein the resource allocation information is determined based on resource requirement information of the target application and the resource information of the multiple nodes, and is used to instruct a target node selected from the multiple nodes to execute the target application; Based on the resource allocation information of the target node, sending a resource allocation request to a resource manager of the cluster; The container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information; The resource demand information includes resource configuration information of at least one container and demand information of nodes in the cluster where the resource to be configured is located; Among them, the demand information for the nodes in the cluster where the resources to be configured are located includes: requirements for the resource type of the node, restrictions on the running container, restrictions on the address information of the node, restrictions on the operating status information of the node, and at least one of the requirements for the idle resources.

7. A resource allocation method, characterized in that: The method comprises: Monitoring resource information of multiple nodes in a cluster, wherein the resource information of the node includes attribute information of the node and container information of the node, and the container information of the node is attribute information of a container running on the node; In response to preset trigger conditions, the resource information of multiple nodes in the cluster is sent to the application manager; In response to the resource allocation request sent by the application manager, allocating target resource information indicating a target node in the resource allocation request to a target application indicated by the resource allocation request; wherein the resources indicated in the resource allocation request are allocated in the form of containers; Among them, the container information of the node includes: the type of container running on the node, the number of containers running on the node and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node and the idle resource information.

8. The method according to claim 7, characterized in that The method further comprises: If the target resource information has been occupied by other applications except the target application, resource allocation failure information is sent to the application manager.

9. A resource allocation method, characterized in that: The method comprises: Obtain resource information of multiple nodes, and display the resource information in a preset format; the resource information of the node includes attribute information and container information of the node, wherein the container information of the node is attribute information of the container running on the node; Receive resource allocation information for a target application submitted by a user, the resource allocation information being determined based on resource requirement information of the target application and the resource information of the multiple nodes, and being used to indicate a target node screened from the multiple nodes to execute the target application, wherein the resource requirement information of the target application includes resource configuration information of at least one container and requirement information of a node in the cluster where the resource to be configured is located; Sending the resource allocation information to an application manager; The container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information; Among them, the demand information for the nodes in the cluster where the resources to be configured are located includes: requirements for the resource type of the node, restrictions on the running container, restrictions on the address information of the node, restrictions on the operating status information of the node, and at least one of the requirements for the idle resources.

10. A resource allocation device, characterized in that: include: A first receiving module is used to receive resource information of multiple nodes in the cluster, where the resource information of the node includes attribute information of the node and container information of the node, wherein the container information of the node is attribute information of a container running on the node; A second receiving module is used to receive resource demand information for a target application sent by a client; the resource demand information includes resource configuration information of at least one container and demand information of a node in the cluster where the resource to be configured is located; A first determining module, configured to determine target resource information of a target node matching the resource demand information based on the resource information of the multiple nodes; A first sending module, configured to send a resource allocation request to a resource manager of the cluster based on the determined target resource information of the target node; The container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information; The requirement information for the nodes in the cluster where the resources to be configured are located includes at least one of the requirement for the resource type of the node, restrictions on the running containers, restrictions on the address information of the node, restrictions on the operating status information of the node, and requirement for idle resources.

11. A resource allocation device, characterized in that: include: A third receiving module is used to receive resource information of multiple nodes in the cluster; The resource information of the node includes attribute information of the node and container information of the node, wherein the container information of the node is attribute information of the container running on the node; A second sending module, configured to respond to a resource allocation instruction for a target application and send resource information of the plurality of nodes to a client; a fourth receiving module, configured to receive resource allocation information for the target application sent by the client, wherein the resource allocation information is determined based on the resource requirement information of the target application and the resource information of the plurality of nodes, and is configured to instruct a target node selected from the plurality of nodes to execute the target application; A third sending module, configured to send a resource allocation request to a resource manager of a cluster based on the resource allocation information of the target node; The container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information; The resource demand information includes resource configuration information of at least one container and demand information of nodes in the cluster where the resource to be configured is located; Among them, the demand information for the nodes in the cluster where the resources to be configured are located includes: requirements for the resource type of the node, restrictions on the running container, restrictions on the address information of the node, restrictions on the operating status information of the node, and at least one of the requirements for the idle resources.

12. A resource allocation device, characterized in that: include: A monitoring module, used to monitor resource information of multiple nodes in the cluster, wherein the resource information of the node includes attribute information of the node and container information of the node, and the container information of the node is attribute information of the container running on the node; A fourth sending module, configured to respond to a preset trigger condition and send resource information of multiple nodes in the cluster to the application manager; an allocation module, configured to respond to the resource allocation request sent by the application manager and allocate target resource information indicating a target node in the resource allocation request to a target application indicated by the resource allocation request; wherein the resources indicated in the resource allocation request are allocated in the form of containers; Among them, the container information of the node includes: the type of container running on the node, the number of containers running on the node and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node and the idle resource information.

13. A resource allocation device, characterized in that: include: An acquisition module, used to acquire resource information of multiple nodes and display the resource information in a preset format; The resource information of the node includes attribute information and container information of the node, wherein the container information of the node is attribute information of the container running on the node; a fifth receiving module, configured to receive resource allocation information for a target application submitted by a user, wherein the resource allocation information is determined based on resource requirement information of the target application and the resource information of the plurality of nodes, and is configured to indicate a target node selected from the plurality of nodes to execute the target application, wherein the resource requirement information of the target application includes resource configuration information of at least one container and requirement information of a node in the cluster where the resource to be configured is located; A fifth sending module, used to send the resource allocation information to the application manager; The container information of the node includes: the type of container running on the node, the number of containers running on the node, and the resource information occupied by the containers running on the node; the attribute information of the node includes: the resource type of the node, the address information of the node, the operation status information of the node, and the idle resource information; Among them, the demand information for the nodes in the cluster where the resources to be configured are located includes: requirements for the resource type of the node, restrictions on the running container, restrictions on the address information of the node, restrictions on the operating status information of the node, and at least one of the requirements for the idle resources.

14. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the resource allocation method described in any one of claims 1 to 5, or claim 6, or claims 7 to 8, or claim 9 are performed.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the resource allocation method described in any one of claims 1 to 5, or claim 6, or claims 7 to 8, or claim 9 are executed.

Images