Resource management method, global resource manager and node resource manager

Abstract

The invention discloses a resource management method, a global resource manager and a node resource manager. The method comprises the following steps: receiving a first resource adjustment request sent by a hybrid management platform, and analyzing a target node and a target resource configuration parameter corresponding to the target node; in response to the fact that the registered resource configuration parameters of the target node are different from the target resource configuration parameters, writing the target node and the target resource configuration parameters into a second resource adjustment request, and forwarding the second resource adjustment request to a node resource manager to which the target node belongs; receiving a returned resource adjustment result, updating a registration resource configuration parameter of the target node and performing persistent storage; and forwarding the resource adjustment result to the hybrid department management platform. According to the method, the resource dynamic compression and expansion on the premise of not restarting the nodes integrally are realized, the offline tasks can be conveniently filled or released on the premise of meeting the resource requirements of the online tasks according to the priorities of the online tasks and the predicted traffic, the resource utilization rate is improved, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a resource management method, a global resource manager, a node resource manager, an electronic processing platform, a computer-readable medium and a program product. Background Art

[0002] Big data tasks include online tasks and offline tasks. To improve resource utilization and reduce costs, the offline co-location technology is used to deploy online tasks and offline tasks on the same physical resources and perform resource allocation and isolation, thereby achieving flexible, efficient and low-cost physical resource utilization and stable and reliable business assurance.

[0003] The existing offline colocation technology does not support dynamic expansion or reduction of node resources. Resource adjustment can only stop the node tasks as a whole and restart them after adjustment. This will affect the ongoing business and make it impossible to dynamically and flexibly schedule node resources according to the business resource usage. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a resource management method, a global resource manager, and a node resource manager, with the aim of achieving dynamic and refined scheduling of big data task resources without restarting the node as a whole and ensuring the continuous operation of the task, thereby improving resource utilization and reducing resource adjustment costs, so as to solve the problems existing in the background technology.

[0005] To achieve the above object, according to a first aspect of an embodiment of the present invention, a resource management method is provided, which is used for a global resource manager, and includes:

[0006] receiving a first resource adjustment request sent by the colocation management platform, and parsing a target node and a corresponding target resource configuration parameter thereof from the first resource adjustment request;

[0007] In response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, writing the target node and the target resource configuration parameters into a second resource adjustment request, and forwarding the second resource adjustment request to a node resource manager to which the target node belongs;

[0008] Receive the resource adjustment result returned by the node resource manager, update the registered resource configuration parameters of the target node according to the resource adjustment result and store them persistently;

[0009] Forward the resource adjustment result to the colocation management platform.

[0010] Further, in response to the registration resource configuration parameter of the target node being different from the target resource configuration parameter, the method further includes:

[0011] Firstly, assign the first backup resource configuration parameter of the target node to the registered resource configuration parameter, and then assign the registered resource configuration parameter to the target resource configuration parameter;

[0012] Updating the registered resource configuration parameters of the target node according to the resource adjustment result and storing them persistently includes:

[0013] In response to the resource adjustment result being successful, deleting the first backup resource configuration parameter, and maintaining the registered resource configuration parameter unchanged and performing persistent storage;

[0014] In response to the resource adjustment result being a failure, the registered resource configuration parameter is assigned to the first backup resource configuration parameter and is persistently stored.

[0015] Further, in response to the registration resource configuration parameter of the target node being different from the target resource configuration parameter, the method further includes:

[0016] Assigning the second backup resource configuration parameter of the target node to the target resource configuration parameter;

[0017] Updating the registered resource configuration parameters of the target node according to the resource adjustment result and storing them persistently includes:

[0018] In response to the resource adjustment result being successful, assigning the registered resource configuration parameter to the second backup resource configuration parameter and storing it persistently;

[0019] In response to the resource adjustment result being a failure, the second backup resource configuration parameter is deleted, and the registered resource configuration parameter is maintained unchanged and persistently stored.

[0020] According to a second aspect of an embodiment of the present invention, a resource management method is provided, which is used for a node resource manager, wherein the node resource manager is communicatively connected to the global resource manager, and comprises:

[0021] receiving a second resource adjustment request sent by the global resource manager, and parsing a target node and its corresponding target resource configuration parameters from the second resource adjustment request;

[0022] Using the target resource configuration parameters, adjusting the resources allocated to the target node by the host of the target node, and waiting for the host to perform the adjustment;

[0023] The resource adjustment result fed back by the host machine after the adjustment is completed is received, and the resource adjustment result is sent to the global resource manager.

[0024] Furthermore, the target resource configuration parameters include maximum CPU usage and maximum memory usage;

[0025] The step of using the target resource configuration parameters to adjust the resources allocated to the target node by the host machine of the target node includes:

[0026] Generate a resource adjustment command for a container corresponding to the target node and running on the host machine according to the maximum CPU usage and the maximum memory usage;

[0027] The resource adjustment command is executed on the host machine.

[0028] According to a third aspect of an embodiment of the present invention, a global resource manager is provided, including:

[0029] A receiving and parsing module, configured to receive a first resource adjustment request sent by the colocation management platform, and parse the first resource adjustment request to obtain a target node and its corresponding target resource configuration parameters;

[0030] a configuration forwarding module, configured to, in response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, write the target node and the target resource configuration parameters into a second resource adjustment request, and forward the second resource adjustment request to a node resource manager to which the target node belongs;

[0031] A configuration persistence module is used to receive the resource adjustment result returned by the node resource manager, update the registered resource configuration parameters of the target node according to the resource adjustment result and store them persistently;

[0032] A result forwarding module is used to forward the resource adjustment result to the colocation management platform.

[0033] According to a fourth aspect of an embodiment of the present invention, there is provided a node resource manager, which is communicatively connected with the global resource manager, and includes:

[0034] According to a fifth aspect of an embodiment of the present invention, there is provided an electronic processing device, comprising:

[0035] one or more processors; and,

[0036] a memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the one or more processors implement the method provided by the first aspect and / or the second aspect of the embodiment of the present invention.

[0038] According to a sixth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method provided by the first aspect and / or the second aspect of the embodiment of the present invention is implemented.

[0039] According to a seventh aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method provided by the first aspect and / or the second aspect of the embodiment of the present invention.

[0040] The embodiments of the invention have the following advantages or beneficial effects: dynamic compression and expansion of node resources are achieved without restarting the node as a whole and ensuring the continuous operation of tasks, laying a foundation for intelligent dynamic real-time management of node resources in offline co-location systems, facilitating intelligent dynamic allocation of resources in offline co-location systems according to the priority of online tasks and predicted traffic, filling or releasing offline tasks on the premise of meeting the resource requirements of online tasks, improving resource utilization and reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0042] Figure 1 is a schematic diagram of the principle of an offline task scheduling system according to an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of the principle of a resource management method according to an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of the main process of a resource management method for a global resource manager according to an embodiment of the present invention;

[0045] Figure 4 is a schematic diagram of main process lanes of a resource management method according to an embodiment of the present invention;

[0046] Figure 5 is a schematic diagram of main process lanes of a resource management method according to another embodiment of the present invention;

[0047] Figure 6 is a schematic diagram of the main process of a resource management method for a node resource manager according to an embodiment of the present invention;

[0048] Figure 7 is a schematic diagram of main modules of a global resource manager according to an embodiment of the present invention;

[0049] Figure 8 is a schematic diagram of main modules of a node resource manager according to an embodiment of the present invention;

[0050] Fig. 9 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0051] Fig.10 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0052] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0053] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0054] In existing technologies, online tasks have high requirements for service level agreements (SLAs), resource utilization has tidal characteristics, and resource usage peaks are usually concentrated during the day; while the running time of some offline tasks can be divided into intervals, and the peak can usually be scheduled at night, and they have the characteristics of high resource utilization, insensitivity to latency, and high fault tolerance. Therefore, online tasks and offline tasks can be deployed in a mixed manner to improve resource utilization and reduce costs.

[0055] However, the existing offline colocation technology resource adjustment can only start or stop the overall node task through the includes file (used to specify the nodes to be added to the cluster) and the excludes file (used to specify the nodes to be deactivated). This method has a large adjustment range and affects the tasks being executed, and cannot achieve dynamic and refined adjustment of node resource usage. In order to solve this problem, the present invention proposes a big data task resource adjustment method, which realizes dynamic and refined compression and expansion of node resources through communication and processing between the colocation management platform, resource manager and node manager, laying the foundation for intelligent dynamic real-time management of node resources in offline colocation systems, facilitating the intelligent dynamic allocation of resources in offline colocation systems according to the priority and predicted traffic of online tasks, filling or releasing offline tasks on the premise of meeting the resource requirements of online tasks, improving resource utilization and reducing costs. 。

[0056] Figure 1Schematic diagram of the principle of an offline task scheduling system according to an embodiment of the present invention. Figure 1 As shown, in the embodiment of the present invention, on the one hand, according to the macro scheduling strategy of the colocation management platform, such as the tidal dynamic scheduling strategy, high-priority offline tasks with low latency and high service level agreement requirements are scheduled to run in the offline cluster A. Low-priority offline tasks that are not sensitive to latency, have high fault tolerance, and allow eviction and restart are scheduled to run in the offline colocation cluster B, using the idle resources of online tasks to fill offline tasks and reduce costs; on the other hand, for a single node running offline tasks in the offline colocation cluster, dynamic and subtle adjustments are made to the occupied resources, including resource compression and resource expansion. Resource compression is to give up occupied resources to online tasks, such as Figure 1 Node N1 in the example, resource expansion is to expand the idle resources occupied by online tasks, such as Figure 1 Node N2 in.

[0057] Figure 2 : is a schematic diagram of the principle of the resource management method according to an embodiment of the present invention. In order to make full use of the offline mixed resources between big data offline tasks and online tasks, in an embodiment of the present invention, the communication and processing between the mixed management platform, the global resource manager (RM, ResourcManager) and the node resource manager (NM, NodeManager) are used to adjust the resources of a single node. Among them, the mixed management platform provides a unified management and scheduling interface for big data cluster computing resources for upper-layer applications, the global resource manager is responsible for the core component of global scheduling and management of cluster resources, and the global resource manager monitors the resource usage on each node through the node resource manager and ensures the effective use of cluster resources. In some specific embodiments of the present invention, containers are used to deploy offline tasks to realize the sharing of online task resources to offline tasks. Specifically, the mixed management platform and the global resource manager (RM) transmit resource configuration parameters through the http communication protocol, and the global resource manager (RM) and the node resource manager (NM) perform asynchronous transmission of resource configuration parameters through the native heartbeat protocol, and the synchronous transmission of resource configuration parameters can also be realized through the http communication protocol. Compared with the former, the latter has shorter latency and can promptly feedback resource adjustment results for real-time synchronization, which is beneficial for the colocation management platform to dynamically and quickly adjust resources and make the next decision.

[0058] Figure 3 FIG. 1 is a schematic diagram of the main process of the resource management method for a global resource manager according to an embodiment of the present invention. Figure 3 As shown, a resource management method for a global resource manager in one embodiment of the present invention includes the following steps S110 to S140.

[0059] Step S110: receiving a first resource adjustment request sent by the colocation management platform, and parsing the first resource adjustment request to obtain a target node and its corresponding target resource configuration parameters.

[0060] Step S120, in response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, writing the target node and the target resource configuration parameters into a second resource adjustment request, and forwarding the second resource adjustment request to a node resource manager to which the target node belongs.

[0061] Step S130: receiving the resource adjustment result returned by the node resource manager, updating the registered resource configuration parameters of the target node according to the resource adjustment result and storing them persistently.

[0062] Step S140: forwarding the resource adjustment result to the colocation management platform, so that the colocation management platform can be informed of the resource configuration status in time to make the next decision.

[0063] It is understandable that, for resource adjustment requests, the terms "first" and "second" are used only for representation purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. The present invention defines that the features defined by "first", "second", etc. may explicitly or implicitly include at least one of the features. Therefore, the first resource adjustment request indicates the target to which it is sent, rather than a restriction on the resource management request.

[0064] Furthermore, the target resource configuration parameters include the maximum usage of the CPU and the maximum usage of the memory, specifically, the maximum occupancy percentage of the CPU and the maximum occupancy percentage of the memory, or the maximum number of CPU cores and the maximum usage of the memory.

[0065] It can be understood that the registered resource configuration parameters are resource configuration parameters that are registered by the node before adjustment and persistently stored in the cluster, and their role is to restore the resource configuration of the node after the node is restarted.

[0066] Specifically, by comparing the registered resource configuration parameters of the target node with the target resource configuration parameters, determine whether the registered resource configuration parameters of the target node are different from the target resource configuration parameters. If they are the same, no resource adjustment is required. If the registered resource configuration parameters are greater than the target resource configuration parameters, resource compression is required; if the registered resource configuration parameters are less than the target resource configuration parameters, resource expansion is required. For the latter two, the target node and the target resource configuration parameters need to be written into a second resource adjustment request, and the second resource adjustment request is forwarded to the node resource manager to which the target node belongs so that the node resource manager can make adjustments.

[0067] Specifically, there are two different processing scenarios for updating the registered resource configuration parameters of the target node according to the resource adjustment result and performing persistent storage.

[0068] The first case, such as Figure 4 As shown, Figure 4 1 is a schematic diagram of the main process of the resource management method according to an embodiment of the present invention. In this embodiment, in response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, the method further includes:

[0069] Step S121a, first assigning the first backup resource configuration parameter of the target node as the registered resource configuration parameter, and then assigning the registered resource configuration parameter as the target resource configuration parameter;

[0070] Correspondingly, updating the registered resource configuration parameters of the target node according to the resource adjustment result and storing them persistently includes:

[0071] Step S131a, in response to the resource adjustment result being successful, deleting the first backup resource configuration parameter, and maintaining the registered resource configuration parameter unchanged and performing persistent storage;

[0072] Step S132a: in response to the resource adjustment result being a failure, assigning the registered resource configuration parameter to the first backup resource configuration parameter and storing it persistently.

[0073] The second situation, such as Figure 5 As shown, Figure 5 1 is a schematic diagram of the main process lanes of a resource management method according to another embodiment of the present invention. In this embodiment, in response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, the method further includes:

[0074] Step S121b, assigning the second backup resource configuration parameter of the target node to the target resource configuration parameter;

[0075] Correspondingly, updating the registered resource configuration parameters of the target node according to the resource adjustment result and storing them persistently includes:

[0076] Step S131b, in response to the resource adjustment result being successful, assigning the registered resource configuration parameter to the second backup resource configuration parameter and storing it persistently;

[0077] Step S132b, in response to the resource adjustment result being a failure, deleting the second backup resource configuration parameter, and maintaining the registered resource configuration parameter unchanged and performing persistent storage.

[0078] Figure 6 FIG. 1 is a schematic diagram of the main process of the resource management method for a node resource manager according to an embodiment of the present invention. Figure 6 As shown, a resource management method for a node resource manager in an embodiment of the present invention includes the following steps S610 to S630: wherein the node resource manager is in communication connection with the global resource manager.

[0079] Step 610: Receive a second node resource adjustment request sent by the global resource manager, and parse the second node resource adjustment request to obtain a target node and its corresponding target resource configuration parameters.

[0080] Step 620: Use the target resource configuration parameters to adjust the resources allocated to the target node by the host of the target node, and wait for the host to perform the adjustment.

[0081] It can be understood that in some embodiments of the present invention, resource isolation adopts a shared kernel and container architecture, and the target resource configuration parameters required for adjustment include maximum CPU usage and maximum memory usage.

[0082] Step 630: receiving the resource adjustment result fed back by the host machine after the adjustment is completed, and sending the resource adjustment result to the global resource manager.

[0083] Furthermore, in some embodiments of the present invention, the step of using the target resource configuration parameters for adjustment includes:

[0084] Generate a resource adjustment command corresponding to the container of the target node running on the host machine according to the maximum CPU usage and the maximum memory usage;

[0085] The resource adjustment command is executed on the host machine.

[0086] By adopting the resource management method of the present invention, without restarting the node as a whole, the target node can be dynamically screened and the target resource configuration parameters can be determined according to the predicted priority of the node's online task and the expected traffic situation, and the resources can be expanded or released in proportion; the resources that should be allocated to offline tasks at each node can also be analyzed and predicted in real time according to the resource occupancy data and system status collected by the monitoring system, usually giving priority to online tasks during the day and offline low-priority tasks at night, and then automatically calling the program interface of the method of the present invention to adjust resources according to a pre-defined execution strategy, and intelligently compressing and expanding resources.

[0087] For example, refer to Figure 1 and Figure 4As shown, the colocation management platform obtains the current registered configuration parameters of node N1, the CPU core usage percentage is 52%, and the memory usage is 186G. It is assumed that the online tasks on the host machine where the node is located will usher in a small peak and resources need to be used first. Therefore, it is necessary to adjust the resources of node N1 in advance and release a part of them for the online tasks of the host machine. Assume that the node N1 is manually set to release half of its resources to online tasks, that is, the target resource configuration parameters: CPU core usage is 26%, and memory usage is 93G. The target resource configuration parameters are passed to the interface provided by the program that applies the method of the present invention, and the program is executed as follows Figure 4 In the process shown, the adjusted node N1 reports a successful adjustment result, that is, the CPU core usage is 26% and the memory usage is 93G. Dynamic compression of resources is completed without restarting the entire node and ensuring the continuous operation of tasks.

[0088] For example, refer to Figure 1 and Figure 5 , the colocation management platform obtains the currently registered configuration parameters of node N2, the CPU core usage percentage is 23%, and the memory usage is 66G. It is assumed that the colocation management platform detects that the online tasks on the host machine where the node is located have entered a low period and there are idle resources available. Therefore, the colocation management platform expands the resources of node N2. It is assumed that an intelligent method is used to calculate that node N2 can occupy resources to the maximum extent, that is, the target resource configuration parameters: CPU core usage is 69%, and memory usage is 198G. The target resource configuration parameters are passed to the interface provided by the program that applies the method of the present invention, and the program is executed as follows Figure 4 In the process shown, the adjusted node N2 reports a successful adjustment result, that is, the CPU core usage is 69% and the memory usage is 198G. The dynamic expansion of resources is completed without restarting the entire node and ensuring the continuous operation of the task.

[0089] According to another aspect of the embodiment of the present invention, Figure 7 As shown, a global resource manager 700 is provided, including:

[0090] A receiving and parsing module, configured to receive a first resource adjustment request sent by the colocation management platform, and parse the first resource adjustment request to obtain a target node and its corresponding target resource configuration parameters;

[0091] a configuration forwarding module, configured to, in response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, write the target node and the target resource configuration parameters into a second resource adjustment request, and forward the second resource adjustment request to a node resource manager to which the target node belongs;

[0092] A configuration persistence module is used to receive the resource adjustment result returned by the node resource manager, update the registered resource configuration parameters of the target node according to the resource adjustment result and store them persistently;

[0093] A result forwarding module is used to forward the resource adjustment result to the colocation management platform.

[0094] According to another aspect of the embodiment of the present invention, Figure 8 As shown, a node resource manager 800 is provided, including:

[0095] A configuration parsing module, configured to receive a second resource adjustment request sent by the global resource manager, and parse a target node and its corresponding target resource configuration parameters from the second resource adjustment request;

[0096] A configuration processing module, configured to use the target resource configuration parameters to adjust the resources allocated to the target node by the host of the target node, and wait for the host to perform the adjustment;

[0097] The result feedback module is used to receive the resource adjustment result fed back by the host machine after the adjustment is completed, and send the resource adjustment result to the global resource manager.

[0098] Fig. 9 An exemplary system architecture 900 is shown to which the resource management method according to an embodiment of the present invention can be applied.

[0099] like Fig. 9 As shown, system architecture 900 may include terminal devices 901, 902, 903, network 904 and server 905. Network 904 is used to provide a medium for communication links between terminal devices 901, 902, 903 and server 905. Network 904 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0100] The user can use the terminal devices 901, 902, 903 to interact with the server 905 through the network 904 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 901, 902, 903, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0101] The terminal devices 901 , 902 , and 903 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0102] The server 905 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 901, 902, and 903. The backend management server may analyze and process the received data such as product information query requests, and feed back the processing results to the terminal device.

[0103] It should be noted that the resource management method provided in the embodiment of the present invention is generally executed by the server 905 , and accordingly, the global resource manager and the node resource manager are generally set in the server 905 .

[0104] It should be understood that Fig. 9 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0105] Reference below Fig.10 , which shows a schematic diagram of the structure of a computer system 1000 of a terminal device suitable for implementing an embodiment of the present invention. Fig.10 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0106] like Fig.10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the system 1000 are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0107] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0108] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1009, and / or installed from the removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above-mentioned functions defined in the system of the present invention are executed.

[0109] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0110] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a matching module, a deduplication module and a reorganization module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the reorganization module may also be described as a "module that reorganizes according to the relationship".

[0112] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0113] receiving a first resource adjustment request sent by the colocation management platform, and parsing a target node and a corresponding target resource configuration parameter thereof from the first resource adjustment request;

[0114] In response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, writing the target node and the target resource configuration parameters into a second resource adjustment request, and forwarding the second resource adjustment request to a node resource manager to which the target node belongs;

[0115] Receive the resource adjustment result returned by the node resource manager, update the registered resource configuration parameters of the target node according to the resource adjustment result and store them persistently;

[0116] Forward the resource adjustment result to the colocation management platform.

[0117] or such that the device comprises:

[0118] receiving a second resource adjustment request sent by the global resource manager, and parsing a target node and its corresponding target resource configuration parameters from the second resource adjustment request;

[0119] Using the target resource configuration parameters, adjusting the resources allocated to the target node by the host of the target node, and waiting for the host to perform the adjustment;

[0120] The resource adjustment result fed back by the host machine after the adjustment is completed is received, and the resource adjustment result is sent to the global resource manager.

[0121] According to the technical solution of the embodiment of the present invention, the present invention has the following advantages or beneficial effects: it realizes dynamic compression and expansion of node resources without restarting the node as a whole and ensuring the continuous operation of tasks, lays the foundation for intelligent dynamic real-time management of node resources in offline co-location systems, facilitates the intelligent dynamic allocation of resources in offline co-location systems according to the priority of online tasks and predicted traffic, fills or releases offline tasks on the premise of meeting the resource requirements of online tasks, improves resource utilization and reduces costs.

[0122] The specific implementation methods described herein do not constitute limitations on the scope of protection of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A resource management method, used in a global resource manager, characterized in that: include: receiving a first resource adjustment request sent by the colocation management platform, and parsing a target node and a corresponding target resource configuration parameter thereof from the first resource adjustment request; In response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, writing the target node and the target resource configuration parameters into a second resource adjustment request, and forwarding the second resource adjustment request to a node resource manager to which the target node belongs; Receive the resource adjustment result returned by the node resource manager, update the registered resource configuration parameters of the target node according to the resource adjustment result and store them persistently; Forward the resource adjustment result to the colocation management platform.

2. The method according to claim 1, characterized in that In response to the registered resource configuration parameter of the target node being different from the target resource configuration parameter, the method further includes: Firstly, assign the first backup resource configuration parameter of the target node to the registered resource configuration parameter, and then assign the registered resource configuration parameter to the target resource configuration parameter; Updating the registered resource configuration parameters of the target node according to the resource adjustment result and storing them persistently includes: In response to the resource adjustment result being successful, deleting the first backup resource configuration parameter, and maintaining the registered resource configuration parameter unchanged and performing persistent storage; In response to the resource adjustment result being a failure, the registered resource configuration parameter is assigned to the first backup resource configuration parameter and is persistently stored.

3. The method according to claim 1, characterized in that In response to the registered resource configuration parameter of the target node being different from the target resource configuration parameter, the method further includes: Assigning the second backup resource configuration parameter of the target node to the target resource configuration parameter; Updating the registered resource configuration parameters of the target node according to the resource adjustment result and storing them persistently includes: In response to the resource adjustment result being successful, assigning the registered resource configuration parameter to the second backup resource configuration parameter and storing it persistently; In response to the resource adjustment result being a failure, the second backup resource configuration parameter is deleted, and the registered resource configuration parameter is maintained unchanged and persistently stored.

4. A resource management method for a node resource manager, wherein the node resource manager is in communication connection with a global resource manager using the method according to any one of claims 1 to 3, characterized in that: include: receiving a second resource adjustment request sent by the global resource manager, and parsing a target node and its corresponding target resource configuration parameters from the second resource adjustment request; Using the target resource configuration parameters, adjusting the resources allocated to the target node by the host of the target node, and waiting for the host to perform the adjustment; The resource adjustment result fed back by the host machine after the adjustment is completed is received, and the resource adjustment result is sent to the global resource manager.

5. The method according to claim 4, characterized in that The target resource configuration parameters include maximum CPU usage and maximum memory usage; The step of using the target resource configuration parameters to adjust the resources allocated to the target node by the host machine of the target node includes: Generate a resource adjustment command for a container corresponding to the target node and running on the host machine according to the maximum CPU usage and the maximum memory usage; The resource adjustment command is executed on the host machine.

6. A global resource manager, characterized in that: include: A receiving and parsing module, configured to receive a first resource adjustment request sent by the colocation management platform, and parse the first resource adjustment request to obtain a target node and its corresponding target resource configuration parameters; a configuration forwarding module, configured to, in response to the registered resource configuration parameters of the target node being different from the target resource configuration parameters, write the target node and the target resource configuration parameters into a second resource adjustment request, and forward the second resource adjustment request to a node resource manager to which the target node belongs; A configuration persistence module is used to receive the resource adjustment result returned by the node resource manager, update the registered resource configuration parameters of the target node according to the resource adjustment result and store them persistently; The result forwarding module is used to forward the resource adjustment result to the colocation management platform.

7. A node resource manager, communicatively connected to the global resource manager according to claim 6, characterized in that: include: A configuration parsing module, configured to receive a second resource adjustment request sent by the global resource manager, and parse a target node and its corresponding target resource configuration parameters from the second resource adjustment request; A configuration processing module, configured to use the target resource configuration parameters to adjust the resources allocated to the target node by the host of the target node, and wait for the host to perform the adjustment; The result feedback module is used to receive the resource adjustment result fed back by the host machine after the adjustment is completed, and send the resource adjustment result to the global resource manager.

8. An electronic processing device, characterized in that: include: one or more processors; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images