A resource management method and device, electronic equipment and medium
By optimizing the scaling method of the node set, selecting idle dynamic nodes based on node price, and pre-configuring the dependent environment and sharing the image, the efficiency and cost issues of node scaling under traffic uncertainty are solved, and more efficient application management is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2023-03-21
- Publication Date
- 2026-05-05
AI Technical Summary
Existing technologies lack targeted node scaling when facing traffic uncertainty, resulting in low application scaling efficiency and high costs.
By selecting or deleting idle dynamic nodes from the node inventory according to the node price in ascending or descending order, and combining the pre-configuration of dependent environments and image sharing, the scaling process of the node set is optimized.
It enables more targeted node scaling, reduces overall costs, improves the efficiency of application scaling and the utilization of dependent environments, and reduces the impact on running applications.
Smart Images

Figure CN116302540B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and more particularly to the field of device management and application management technology. Background Technology
[0002] With the increasing popularity of serverless technology, more and more platform products based on serverless technology have emerged. These products can provide functions computing services, fully managed application services, or elastic container services. Among them, fully managed application services are based on the container orchestration engine (Kubernetes) and container technology, which can host applications for users and adaptively scale applications elastically according to the amount of traffic. Summary of the Invention
[0003] This disclosure provides a resource management method, apparatus, electronic device, and medium.
[0004] A first aspect of this disclosure provides a resource management method, including:
[0005] Determine whether the node set needs to be expanded or reduced;
[0006] If it is determined that the node set needs to be expanded, nodes are selected from the node inventory in order of price from low to high. The selected nodes are added to the node set as idle dynamic nodes until the number of idle dynamic nodes in the node set is a preset number. In this case, no business applications are running in the nodes in the node inventory or in the idle dynamic nodes.
[0007] If it is determined that the node set needs to be scaled down, then nodes are deleted from the idle dynamic nodes of the node set in descending order of price until the number of idle dynamic nodes in the node set is the preset number.
[0008] A second aspect of this disclosure provides a resource management device, comprising:
[0009] The determination module is used to determine whether the node set needs to be expanded or reduced in size.
[0010] The node expansion module is used to select nodes from the node inventory in order of price from low to high if the determining module determines that the node set needs to be expanded. The selected nodes are added to the node set as idle dynamic nodes until the number of idle dynamic nodes in the node set is a preset number. In this case, no business applications are running in the nodes in the node inventory or in the idle dynamic nodes.
[0011] The node scaling-down module is used to delete nodes from the idle dynamic nodes of the node set in descending order of price if the determining module determines that node scaling-down is required, until the number of idle dynamic nodes in the node set is the preset number.
[0012] A third aspect of this disclosure provides an electronic device, including:
[0013] At least one processor; and
[0014] A memory communicatively connected to the at least one processor; wherein,
[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in any one of the first aspects.
[0016] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any one of the first aspects.
[0017] A fifth aspect of this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the method according to any one of the first aspects.
[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0019] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0020] Figure 1 This is a flowchart of a resource management method provided in an embodiment of this disclosure;
[0021] Figure 2 This is a flowchart of an application migration method provided in an embodiment of this disclosure;
[0022] Figure 3 This is a flowchart of a method for configuring a dependent environment provided in an embodiment of this disclosure;
[0023] Figure 4 This is an exemplary schematic diagram of a configuration dependent environment process provided in an embodiment of this disclosure;
[0024] Figure 5This is an exemplary schematic diagram of a mirror sharing process provided in an embodiment of this disclosure;
[0025] Figure 6 This is a flowchart illustrating a method for creating a copy according to an embodiment of this disclosure;
[0026] Figure 7 This is an exemplary schematic diagram of a resource management process provided in an embodiment of this disclosure;
[0027] Figure 8 This is a flowchart of a node scaling method provided in an embodiment of this disclosure;
[0028] Figure 9 This is a schematic diagram of the structure of a resource management device provided in an embodiment of this disclosure;
[0029] Figure 10 This is a block diagram of an electronic device used to implement the resource management method of the embodiments of this disclosure. Detailed Implementation
[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0031] Typically, managed applications have large codebases and require significant runtime resources. To reduce response latency caused by application cold starts, N application instances are usually pre-configured in the node set. However, due to the uncertainty of traffic, especially in scenarios with significant traffic peaks and valleys, simply scaling up or down the application may not be sufficient to meet the demand in the face of sudden increases or decreases in traffic. In such cases, it is necessary to scale up or down the nodes in the node set.
[0032] Currently, when scaling up or down nodes, nodes are typically randomly selected to join the node set or randomly deleted from the node set. This method involves uniform scheduling of all nodes in the set, resulting in a lack of targeted scaling up and down.
[0033] To enable more targeted scaling of nodes, this disclosure provides a resource management method. This method can be applied to electronic devices, such as directly to electronic devices, virtual machines within electronic devices, or containers within electronic devices; this disclosure does not specifically limit its application. The electronic device can be a server or desktop computer, or other device with data processing capabilities.
[0034] like Figure 1 As shown in the embodiments of this disclosure, the resource management method includes the following steps:
[0035] S101. Determine whether the node set needs to be expanded or shrunk. If it is determined that the node set needs to be expanded, then execute S102; if it is determined that the node set needs to be shrunk, then execute S103.
[0036] In this embodiment, it can be periodically determined whether the number of idle dynamic nodes in the node set is a preset number. For example, every t minutes, it is determined whether the number of idle dynamic nodes in the node set is N. If the number of idle dynamic nodes is less than the preset number, it is determined that the node set needs to be expanded; if the number of idle dynamic nodes exceeds the preset number, it is determined that the node set needs to be shrunk. The preset number can be set according to requirements.
[0037] Alternatively, the electronic device can determine whether to expand or shrink the node set based on the received instructions. Other methods may also be used, which are not specifically limited in this embodiment.
[0038] In this embodiment of the disclosure, no business application is running within the idle dynamic node. The business application may be a managed application specified by the user. Furthermore, other applications, such as system applications, may run within the idle dynamic node.
[0039] S102. Select nodes from the node inventory in order of price from low to high, treat the selected nodes as idle dynamic nodes, and add them to the node set until the number of idle dynamic nodes in the node set is the preset number.
[0040] When expanding the node capacity, the node with the lowest price can be searched in the node inventory first. If found, the node is designated as an idle dynamic node and added to the node set. If it does not exist or the number of idle dynamic nodes does not reach the preset number, the node with the second lowest price can be searched in the node inventory, and so on, until the number of idle dynamic nodes in the node set reaches the preset number.
[0041] None of the nodes in the node inventory are running business applications. Since the nodes in the node inventory are running business applications, when a node is added to the node set from the node inventory, it can be considered as an idle dynamic node in the node set.
[0042] After expanding the node set, the number of idle dynamic nodes in the node set is maintained at a preset number. Compared with the method of expanding nodes by predicting the number of nodes required for traffic, this method is prone to insufficient expansion nodes in scenarios with sudden traffic surges, affecting the efficiency of application expansion. However, the embodiment of this disclosure can maintain the number of idle dynamic nodes at a preset number after expansion, ensuring that there are enough idle dynamic nodes to deploy copies of the business application when expanding the application later, thereby guaranteeing the efficiency of application expansion.
[0043] S103. Delete nodes from the idle dynamic nodes in the node set in descending order of price until the number of idle dynamic nodes in the node set is the preset number.
[0044] After scaling down the node set, the number of idle dynamic nodes in the node set remains at a preset number, which can reduce the number of idle nodes in the node set and reduce the overall cost of the node set.
[0045] Using the above method, this embodiment of the disclosure identifies idle dynamic nodes from the node set. When node expansion is needed, nodes are selected from the node inventory in ascending order of node price, and these selected nodes are added to the node set as idle dynamic nodes. Conversely, when node reduction is needed, nodes are selected from the idle dynamic nodes in descending order of node price and removed from the node set. In other words, this embodiment of the disclosure can expand or shrink the node set based on node price, thereby reducing the overall cost of the node set. Moreover, during expansion or reduction, only idle dynamic nodes that are not running business applications are expanded or reduced, thus minimizing the impact of node expansion or reduction on running business applications. Therefore, more targeted node expansion or reduction is achieved.
[0046] The resource management method provided in the embodiments of this disclosure will be described in detail below:
[0047] In this embodiment, the nodes in the node set can be divided into two parts: fixed nodes and dynamic nodes. Fixed nodes are devices that remain unchanged in the node set and have lower performance; for example, fixed nodes may be out of warranty or old devices. Dynamic nodes are devices that can be added to or removed from the node set and have higher performance; for example, dynamic nodes may be high-specification or high-configuration devices. The price of fixed nodes is lower than that of dynamic nodes.
[0048] Based on the operational status of the nodes, the node set can be divided into five categories: idle fixed nodes, used fixed nodes, idle dynamic nodes, used dynamic nodes, and nodes in preparation.
[0049] In this embodiment of the disclosure, both the used fixed nodes and the used dynamic nodes included in the node set are running business applications, and the running business applications can be the same or different; neither the idle fixed nodes nor the idle dynamic nodes included in the node set are running business applications. Moreover, the price of the used fixed nodes is lower than the price of the used dynamic nodes, and the price of the idle fixed nodes is lower than the price of the idle dynamic nodes.
[0050] Among them, idle fixed nodes can be recorded as idle fixed resources (IDEL_FIX_RESOURSE), used fixed nodes can be recorded as occupied fixed resources (BUSY_FIX_RESOURCE), idle dynamic nodes can be recorded as idle dynamic resources (IDLE_DYNAMIC_RESOURCE), used dynamic nodes can be recorded as occupied dynamic resources (BUSY_DYNAMIC_RESOURSE), and nodes in preparation can be recorded as resources in preparation (PREPARING_RESOURCE).
[0051] This disclosure embodiment can record information about each node within a node set in a database. For example, in a Remote Dictionary Server (Redis), information about each idle fixed node, idle dynamic node, and node in preparation is recorded in a set structure, and information about each used fixed node and used dynamic node is recorded in an ordered list (zset) structure. The key recorded for each node in Redis represents the node type it belongs to in the node set. The members recorded in Redis include: Internet Protocol (IP) and deployment dependencies. The value recorded in Redis includes the node's remaining memory.
[0052] In this disclosure embodiment, see Figure 2 After expanding the node set in S102 above, application migration can also be performed. The application migration process includes the following steps:
[0053] S201. Determine if the target node exists among the idle dynamic nodes added to the node set during this node expansion. If it exists, proceed to S202.
[0054] Specifically, the target node is defined as an idle dynamic node whose price is lower than that of a used dynamic node. In other words, the target node is an idle dynamic node added to the node set during this node expansion that has a price lower than that of a used dynamic node.
[0055] If the prices of all idle dynamic nodes added to the node set during this node expansion are higher than those of the used dynamic nodes, meaning there is no target node, it indicates that the prices of the newly added nodes during this node expansion are too high. In order to save costs, it is not appropriate to migrate the current application replicas in the used dynamic nodes to the newly added nodes. In this case, no application migration will be performed, that is, the application migration of the used dynamic nodes will not be performed in accordance with the methods of S202 and S203.
[0056] S202. Migrate the business application replicas in the used dynamic nodes to the target node.
[0057] Optionally, if there are multiple used dynamic nodes in the node set, when executing S201, for each used dynamic node, it can be determined whether the price of the used dynamic node is higher than the price of any newly added idle dynamic node, in descending order of price. If so, when executing S202, the business application replicas in the used dynamic node are migrated to the newly added idle dynamic node.
[0058] If there are multiple target nodes, meaning that the prices of multiple newly added idle dynamic nodes are lower than those of the used dynamic nodes, then the business application replicas in the used dynamic nodes will be migrated to the node with the lowest price among the target nodes.
[0059] S203. Update the used dynamic nodes to idle dynamic nodes, and update the target node to a used dynamic node.
[0060] Since the business application replicas within the used dynamic node have been migrated to the target node, the business application is no longer running within the used dynamic node. Therefore, the used dynamic node is updated to an idle dynamic node.
[0061] Since a copy of the business application has been migrated within the target node, meaning that the business application is running within the target node, it is necessary to update the target node from an idle dynamic node to a used dynamic node.
[0062] Using the above method, embodiments of this disclosure can migrate copies of business applications from used dynamic nodes to idle dynamic nodes with lower prices, so that business applications run on nodes with lower prices as much as possible, thereby reducing the cost of running business applications.
[0063] In this embodiment of the disclosure, before performing application migration in S201-S203, the target copy, which is the first copy created by the currently running business application, can be migrated first. The target copy and non-target copy can be distinguished by a tag field. For example, the tag field for the target copy is: replica_order=1; the tag field for the non-target copy is: replica_order=0.
[0064] See Figure 2The migration process for the target copy includes the following steps:
[0065] S204. Determine if the target replica is running in the currently used dynamic nodes. If yes, proceed to S205. If no, do not perform application migration, i.e., do not migrate to the replica initially created by the business application.
[0066] In this embodiment of the disclosure, there may be multiple currently running business applications in the node set, and each business application has an initial copy.
[0067] If there are multiple target replicas, the target replicas can be sorted according to the required memory of each target replica, and S205-S207 can be executed for each target replica in sequence.
[0068] S205. Determine whether the remaining memory of the used fixed nodes is greater than or equal to the rated memory required by the target replica. If it is greater than or equal to, proceed to S206; if it is less than, proceed to S207.
[0069] S206. Migrate the target copy to the existing fixed node.
[0070] If there are multiple fixed nodes already in use, it can be determined whether there is any remaining memory on the fixed nodes that is greater than or equal to the rated memory required by the target replica. If so, the target replica is migrated to the fixed node with remaining memory greater than or equal to that rated memory.
[0071] If there are multiple used fixed nodes with remaining memory greater than or equal to the rated memory, then the target replica will be migrated to the used fixed node with the most remaining memory among these multiple used fixed nodes.
[0072] S207. Migrate the target replica to an idle fixed node and update the idle fixed node to a used fixed node.
[0073] Optionally, the target replica can be migrated to any idle fixed node, or the target replica can be migrated to the idle fixed node with the most remaining memory.
[0074] Since the idle fixed node was migrated to the target replica, meaning that a business application was running on the idle fixed node, the idle fixed node was updated to a used fixed node.
[0075] In this embodiment of the disclosure, if after the target copy is migrated out of the used dynamic node, there are no other business application copies in the used dynamic node, then the used dynamic node can be updated to an idle dynamic node.
[0076] Using the above method, embodiments of this disclosure can migrate target replicas from used dynamic nodes to used or idle fixed nodes, thereby reducing the price of nodes running target replicas. Since the target replica is the first replica created by the business application, its lifecycle is relatively stable, meaning it is the least likely to be deleted during application scaling. Therefore, if the target replica is within a used dynamic node, it is difficult for that used dynamic node to be updated to an idle dynamic node, and even more difficult to be removed from the node set. However, embodiments of this disclosure move the target replica to fixed nodes, thereby increasing the possibility of used dynamic nodes being updated to idle dynamic nodes, and further increasing the possibility of being removed from the node set. Since the price of fixed nodes is lower than that of dynamic nodes, embodiments of this disclosure reduce the overall cost of the node set.
[0077] After expanding the node set in S102 above, dependency environments can be pre-configured in the newly added idle dynamic nodes. Multiple dependency environments exist, and different applications can use the same dependency environment; therefore, dependency environments can also be called common dependencies. Each dependency environment can record pre-configured system tools and runtime dependencies, as well as the name of the application it belongs to, the preset number of copies of the application, the dependency environment type, and the dependency environment name.
[0078] See Figure 3 The method for configuring the dependent environment includes the following steps:
[0079] S301. Based on the configuration volume of various preset dependent environment types counted in the current statistical period, filter the first preset number of target types in descending order of configuration volume.
[0080] Preset dependency environment types can include computer programming languages (Python), programming languages (Java), numerical computing extensions (NumPy), data analysis tools (Pandas), and symbolic mathematics systems (TensorFlow), etc. Each dependency environment type can include multiple dependency environments. For example, the Java type includes dependency environments for versions java:10.0.0.1 and java:10.0.0.2.
[0081] The size of the statistical period can be set according to actual needs, such as using one day as a statistical period.
[0082] S302. For each target type, based on the configuration quantity of each dependent environment of that target type in the current statistical period, filter the second preset number of dependent environments in descending order of configuration quantity.
[0083] For example, the Java type includes three versions of dependency environments: java:10.0.0.1, java:10.0.0.2, and java:10.0.0.3. The configuration quantity for java:10.0.0.1 is 10, the configuration quantity for java:10.0.0.2 is 20, and the configuration quantity for java:10.0.0.3 is 30. When the second preset quantity is 2, java:10.0.0.2 and java:10.0.0.3 are selected.
[0084] S303. Based on the proportion of configuration quantities of various target types within the current statistical period, group the idle dynamic nodes added to the node set during this node expansion.
[0085] Assuming that there are 30 idle dynamic nodes added to the node set during this node expansion, and the target types are Java and Python, with 100 Java nodes and 50 Python nodes, then the ratio of target type configurations is 2:1. The 30 idle dynamic nodes are then grouped according to this 2:1 ratio, resulting in two groups of 20 nodes each.
[0086] S304. For each group of idle dynamic nodes, configure a second preset number of dependency environments of a specified type within that group of idle dynamic nodes. The specified type is the target type corresponding to the proportion item to which that group of idle dynamic nodes belongs.
[0087] For example, see Figure 4 Assuming the ratio of Python to Java configurations is 3:1, and that the newly added idle dynamic nodes during this node expansion are node1, node2, node3, and node4, the four idle dynamic nodes are grouped in a 3:1 ratio, resulting in one group of three and another group of one. The three idle dynamic nodes in one group will install the second-highest preset number of Python dependencies. The one idle dynamic node in the other group will install the second-highest preset number of Java dependencies. When configuring each dependency on each node, the dependency will be searched for and installed from the frequently installed dependency libraries.
[0088] When installing dependencies, you can use the corresponding installation tools for each dependency, and you can pre-set a unified installation path for each dependency.
[0089] By employing the above method, embodiments of this disclosure can pre-install the dependency environment for newly added nodes in the node set. This allows the pre-installed dependency environment to be directly used when creating application replicas on those nodes, reducing the possibility of nodes downloading dependency environments during application replica creation and improving the efficiency of application scaling. Furthermore, embodiments of this disclosure decouple the dependency environment from the application image, which not only improves application deployment speed but also allows developers to focus more on the application itself.
[0090] Moreover, when pre-installing dependencies on nodes, all dependencies are typically pre-installed on all nodes. However, application replicas running on the nodes may not be able to use all dependencies, resulting in low utilization of common dependencies.
[0091] The embodiments disclosed herein can group newly added nodes according to the proportion of dependent environment types, and configure dependent environments with a large number of configurations of that type in each group, thereby performing more targeted dependency pre-configuration and improving dependency utilization.
[0092] It should be noted that when creating a business application replica within a node, you can first check whether the node has configured the required dependency environment for that replica. If so, you can directly mount the dependency environment to the specified path of the business application replica so that the business application replica can use the dependency environment at runtime.
[0093] If not, the node can directly download the dependency environment via the network and update its own configured dependency environment statistics table.
[0094] In this embodiment of the disclosure, when configuring the dependent environment in an idle dynamic node, it can be updated to a node in preparation, and after the configuration is completed, the node can be updated to an idle dynamic node.
[0095] In this embodiment of the disclosure, after pre-configuring the dependency environment for the newly added idle dynamic nodes in the node set, the image storage path of each group of idle dynamic nodes can be mounted to a distributed file system. This allows the group of idle dynamic nodes to obtain the image of the business application from the distributed file system when creating a copy of the business application, even if the image of the business application is not stored locally. The image is used to create the copy of the business application.
[0096] This embodiment of the disclosure can also divide each node in the node set into availability zones, with each availability zone corresponding to a distributed file system. In this case, the image storage path of each idle dynamic node after pre-configuring the dependent environment can be mounted to the distributed file system of the availability zone to which the node belongs.
[0097] Optionally, availability zones can be divided according to geographical region or according to data center, etc. This disclosure does not specifically limit this.
[0098] Subsequently, when creating a replica of the business application in a node, it can first check if an image of the business application exists locally. If not, it retrieves the image from the distributed file system. If the image is not found in the distributed file system, it downloads the image from an image repository over the network.
[0099] For example, see Figure 5 In the dynamic resource pool of the node set, that is, in the dynamic nodes of the node set, it is divided into availability zone A and availability zone B. Each availability zone corresponds to a distributed file system. Each node in the availability zone represents an idle dynamic node or a used dynamic node. In availability zone A, node1 runs a replica of APP1 and a replica of APP2, so node1 is a used dynamic node. In availability zone A, node2, as well as in availability zone B, node3 and node4 do not run business applications, so they are all idle dynamic nodes.
[0100] See Figure 5 The image storage paths for nodes 1 and 2 are mounted on the distributed file system of Availability Zone A. Therefore, image 1 of APP1 and image 2 of APP2 stored in node 1 can be shared with node 2. If node 2 needs to create a copy of APP1 or APP2, it can directly obtain the corresponding image from the distributed file system of Availability Zone A. Alternatively, if node 2 needs to create a copy of APP3, and the distributed file system of Availability Zone A does not have an image of APP3, node 2 can remotely pull the image from the image repository.
[0101] Through the above method, the embodiments of this disclosure can utilize a distributed file system to realize image sharing between nodes, reduce the situation where the required image is downloaded from the image repository via the network when the node does not store the required image locally, thereby achieving image acceleration and improving the speed of creating business application replicas in the node.
[0102] Moreover, existing image acceleration methods, such as dynamic loading or peer-to-peer (P2P) acceleration, all require secondary development and adaptation based on the application image, which is a complex process with high adaptation costs.
[0103] The embodiments disclosed herein can directly share application images between nodes based on a distributed file system, without the need for secondary development and adaptation, simplifying the image acceleration process and reducing adaptation costs.
[0104] In this embodiment of the disclosure, the electronic device can record the name of each image in the distributed system, the time of its most recent use, the number of times it has been used, and the name of the distributed file system to which the image belongs.
[0105] This allows electronic devices to manage the distributed file system based on recorded information. Specifically, the electronic device can also detect whether the amount of mirrored data in the distributed file system has reached a threshold. If so, it deletes unused mirrors and mirrors whose last usage time is more than a duration threshold.
[0106] Among them, electronic devices can be periodically or periodically tested to see if the amount of mirrored data in the distributed file system has reached a threshold.
[0107] If the amount of mirrored data in the distributed file system does not reach the threshold, wait for the next check.
[0108] By dividing each node in the node set into availability zones, such that each availability zone corresponds to a distributed file system, electronic devices can manage the distributed file system of each availability zone separately.
[0109] By using the above method, the embodiments of this disclosure can delete unused and low-usage images in the distributed file system, thereby storing high-usage images in the distributed file system. This enables the storage of higher-usage images with limited storage space, improving the practicality of sharing images between nodes.
[0110] In this embodiment of the disclosure, the management of copies of business applications can also be performed. When an electronic device determines that a copy of a business application needs to be created for the first time, it can sequentially determine whether the remaining memory of a node is greater than or equal to the rated memory required for the copy, in the order of used fixed nodes, idle fixed nodes, used dynamic nodes to idle dynamic nodes, until it is determined that the remaining memory of any node is greater than or equal to the rated memory required for the copy, and then create the copy in that node.
[0111] That is, when creating a replica of the business application for the first time, assuming the required memory for the replica is m, first determine whether there exists any used fixed node with remaining memory greater than or equal to m; if so, create the replica within that used fixed node. Then update the remaining memory of that used fixed node by m.
[0112] If the remaining memory of all used fixed nodes is less than m, then determine if there exists any idle fixed node with remaining memory greater than or equal to m; if so, create the replica within that idle fixed node. Then update that idle fixed node as a used fixed node and update its remaining memory by m.
[0113] If the remaining memory of all idle fixed nodes is less than m, then determine if there exists any used dynamic node with remaining memory greater than or equal to m; if so, create the replica within that used dynamic node. Then update the remaining memory of that used dynamic node by m.
[0114] If the remaining memory of all used dynamic nodes is less than m, then determine if there exists any idle dynamic node with remaining memory greater than or equal to m; if so, create the replica within that idle dynamic node. Then update the idle dynamic node as a used dynamic node and update its remaining memory by m.
[0115] Optionally, if the remaining memory of all idle dynamic nodes is less than m, a period of time can be waited before re-evaluating whether any idle dynamic node has a remaining memory greater than or equal to m. Since new idle dynamic nodes may be added during node expansion within the waiting period, this re-evaluation after a certain time improves the success rate of replica creation.
[0116] Using the above method, embodiments of this disclosure can select nodes and create the first copy of a business application in the order of used fixed nodes, idle fixed nodes, used dynamic nodes to idle dynamic nodes, thereby prioritizing creation within fixed nodes with lower prices and reducing the overall cost of the node set.
[0117] In scenarios where performance requirements are not high or traffic peaks and valleys are relatively smooth, using high-performance dynamic nodes to run replicas is not cost-effective. Choosing lower-performance fixed nodes can still meet application needs and reduce costs.
[0118] In this embodiment of the disclosure, the electronic device can also expand the capacity of currently running business applications in the node cluster. See also Figure 6 The capacity expansion method includes the following steps:
[0119] S601. If it is determined that a currently running business application needs to be expanded, for each replica that needs to be expanded from the business application, determine whether the used dynamic nodes meet the first configuration condition. If they do, proceed to S602; otherwise, proceed to S603.
[0120] The first configuration condition includes: the node's remaining memory is greater than or equal to the rated memory required by the replica, and the node's configured dependency environment includes the dependency environment required by the replica.
[0121] When there are multiple used dynamic nodes, it can be determined whether each used dynamic node meets the first configuration condition.
[0122] S602. Create the copy in the existing dynamic node.
[0123] If there are multiple used dynamic nodes that meet the first configuration condition, the replica can be created on any one of these used dynamic nodes. Alternatively, the replica can be created on the used dynamic node with the most remaining memory.
[0124] S603. Determine whether there is a first idle dynamic node that meets the first configuration condition. If there is a first idle dynamic node, then execute S604.
[0125] When there are multiple idle dynamic nodes, it can be determined whether each idle dynamic node meets the first configuration condition.
[0126] S604. Create the copy in the first idle dynamic node.
[0127] If there are multiple first idle dynamic nodes, the replica can be created on any one of them. Alternatively, the replica can be created on the first idle dynamic node with the most remaining memory.
[0128] Using the above method, this embodiment of the present disclosure can create the replica in a node with remaining memory greater than or equal to the rated memory required by the replica and configured with the dependent environment required by the replica, thereby reducing the failure of replica creation and avoiding the need to download the dependent environment over the network in real time when creating the replica, thus improving the efficiency of application scaling.
[0129] In this disclosure embodiment, see Figure 6 After determining in S603 whether there is a first idle dynamic node that meets the first configuration condition, if there is no first idle dynamic node, the electronic device may perform the following steps:
[0130] S605. Determine whether the used dynamic nodes meet the second configuration condition. If they do, proceed to S606; otherwise, proceed to S607.
[0131] The second configuration condition includes: the node's remaining memory is greater than or equal to the rated memory required by the replica.
[0132] S606. Create the copy in the existing dynamic node.
[0133] S607. Determine if a second idle dynamic node exists that meets the second configuration condition. If a second idle dynamic node exists, proceed to S608. If a second idle dynamic node does not exist, creation fails.
[0134] S608. Create the copy in the second idle dynamic node.
[0135] By using the above methods, the configuration requirements for expanding replicas can be reduced, thereby improving the success rate of replica expansion.
[0136] In this embodiment of the disclosure, the electronic device can also scale down copies of the service application. The scaling down method can be implemented by deleting a specified number of copies of the service application from each specified set of copies when it is determined that a currently running service application needs to be scaled down.
[0137] In this case, the dependency environment configured on the node where the specified replica is located is different from the dependency environment required by the replica of the business application.
[0138] When the specified number of replicas is less than the specified reduction number, in addition to deleting each specified replica, other replicas of the application can also be deleted, where the number of deletions is (specified reduction number - specified number of replicas).
[0139] For example, if a business application requires Python 10.0.0.3, and assuming that the business application has 4 replicas, which are evenly configured on node 1 with Python 10.0.0.3 and node 2 with Java 10.0.0.2, then the replica of the business application should be removed from node 2 first.
[0140] When deleting a copy of a business application, electronic devices can directly call the Kubernetes Application Programming Interface (API) to allow Kubernetes to delete the copy.
[0141] Using the above method, embodiments of this disclosure can preferentially delete copies from nodes that have not been configured with the application's required environment, thereby achieving application scaling down and improving the utilization rate of the dependent environment in the node.
[0142] like Figure 7 As shown, the resource management method provided in this embodiment can be implemented by four modules: a scheduling module, a resource control module, a dependency acceleration module, and an image acceleration module.
[0143] The node set and Infrastructure as a Service (IaaS) constitute a resource pool. The node set includes a dynamic resource pool and a fixed resource pool. The dynamic resource pool includes used and idle dynamic nodes, and the fixed resource pool includes used and idle fixed nodes. The IaaS is a node inventory, in which all nodes are high-performance dynamic nodes that are not running business applications.
[0144] The resource control module is used to scale nodes up and down. When scaling nodes up or down, the resource control module can identify idle dynamic nodes from the node information stored in Redis and delete some idle dynamic nodes when scaling down; when scaling up nodes, it records the newly added nodes as idle dynamic nodes in Redis.
[0145] The scheduling module is used for initial application replica creation, application scaling, and application migration. Based on the node types recorded in Redis, the scheduling module can create or migrate business application replicas, and update the node types recorded in Redis after creation or migration.
[0146] The dependency acceleration module is used to pre-configure the dependency environment in the idle dynamic nodes newly added to the node set after node expansion.
[0147] The image acceleration module is used to mount the image storage path of nodes whose dependent environment configuration is completed to the distributed file system of the corresponding availability zone to achieve image acceleration.
[0148] See Figure 8 The following describes the node scaling process in this embodiment of the present disclosure in detail, taking into account practical application scenarios:
[0149] S801. At every interval t, determine the current number of idle nodes num in the node set.
[0150] S802. Determine whether num is less than N or greater than N. If num is less than N, execute S803; if num is greater than N, execute S810.
[0151] S803: Get the node corresponding to the lowest current price in IaaS.
[0152] It is understandable that there may be multiple nodes with the same price. This application starts by acquiring nodes with the lowest price and records the number of nodes acquired so far as c.
[0153] S804. Update the number of nodes X that need to be expanded.
[0154] When S804 is executed for the first time, X = N - num; when S804 is executed for the second time, X = N - num - c, and so on.
[0155] S805. Determine if the number of nodes c obtained is less than X. If yes, proceed to S806; otherwise, proceed to S807.
[0156] S806. The acquired node is treated as an idle dynamic node and added to the node set. The acquired node is then updated to a node in preparation. At this point, the acquired node is removed from the IaaS, added to the node set, and the process returns to S803, thereby further acquiring the node corresponding to the lowest current price in the IaaS.
[0157] S807. Select X nodes from the acquired nodes as idle dynamic nodes and add them to the node set. Update the selected nodes as nodes in preparation. At this point, remove the selected X nodes from the IaaS and add them to the node set.
[0158] S808: Install common dependencies for nodes in preparation.
[0159] S809. Update the node in preparation to an idle dynamic node.
[0160] S810. Sort the idle dynamic nodes in the node set in descending order of price.
[0161] S811. Get the first (num-N) idle dynamic nodes.
[0162] S812. Remove the acquired idle dynamic nodes from the node set.
[0163] The collection, storage, use, processing, transmission, provision, and disclosure of node information and application information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0164] Based on the same inventive concept, and corresponding to the above method embodiments, this disclosure provides a resource management device, such as... Figure 9 As shown, the device includes: a determination module 901, a node expansion module 902, and a node reduction module 903;
[0165] Module 901 is used to determine whether the node set needs to be expanded or shrunk.
[0166] The node expansion module 902 is used to select nodes from the node inventory in order of price from low to high if the determination module 901 determines that the node set needs to be expanded. The selected nodes are added to the node set as idle dynamic nodes until the number of idle dynamic nodes in the node set is a preset number. In this case, no business applications are running in the nodes in the node inventory or in the idle dynamic nodes.
[0167] The node shrinking module 903 is used to delete nodes from the idle dynamic nodes of the node set in descending order of price if the determining module 901 determines that the node set needs to be shrunk, until the number of idle dynamic nodes in the node set is the preset number.
[0168] In some embodiments of this disclosure, the node set further includes used dynamic nodes, in which business applications are running; the apparatus may also include:
[0169] The judgment module is used to determine whether there is a target node among the idle dynamic nodes added to the node set during this node expansion, after the selected node is treated as an idle dynamic node and added to the node set until the number of idle dynamic nodes in the node set reaches a preset number. The price of the target node is lower than the price of the used dynamic nodes.
[0170] The migration module is used to migrate the business application replicas in the used dynamic node to the target node if the judgment result of the judgment module is that the application exists.
[0171] The update module is used to update used dynamic nodes to idle dynamic nodes and to update target nodes to used dynamic nodes.
[0172] In some embodiments of this disclosure, the node set further includes used fixed nodes and idle fixed nodes. Used fixed nodes run business applications, while idle fixed nodes do not. The price of used fixed nodes is lower than the price of used dynamic nodes, and the price of idle fixed nodes is lower than the price of idle dynamic nodes. The apparatus may also include:
[0173] The judgment module is also used to determine whether there is a target replica running in the used dynamic nodes before determining whether there is a target node among the idle dynamic nodes added to the node set during this node expansion. The target replica is the first replica created for the currently running business application.
[0174] The judgment module is also used to determine, if so, whether the remaining memory of the used fixed node is greater than or equal to the rated memory required by the target replica;
[0175] The migration module is also used to migrate the target copy to the used fixed node if the judgment result of the judgment module is greater than or equal to the target copy.
[0176] The migration module is also used to migrate the target replica to an idle fixed node and update the idle fixed node to a used fixed node if the judgment result of the judgment module is less than 1.
[0177] In some embodiments of this disclosure, the device may further include:
[0178] The filtering module is used to select nodes as idle dynamic nodes and add them to the node set until the number of idle dynamic nodes in the node set reaches a preset number. Then, based on the configuration volume of various preset dependent environment types counted in the current statistical period, the first preset number of target types are filtered in descending order of configuration volume.
[0179] The filtering module is also used to filter a second preset number of dependent environments for each target type, based on the configuration volume of each dependent environment of that target type in the current statistical period, in descending order of configuration volume.
[0180] The grouping module is used to group the idle dynamic nodes added to the node set during this node expansion according to the proportion of configuration of various target types in the current statistical period.
[0181] The configuration module is used to configure a second preset number of dependency environments of a specified type within each group of idle dynamic nodes, wherein the specified type is the target type corresponding to the proportion item to which the group of idle dynamic nodes belongs.
[0182] In some embodiments of this disclosure, the device may further include:
[0183] The mounting module is used to mount the image storage path of each group of idle dynamic nodes to a distributed file system after configuring a second preset number of dependency environments of a specified type within that group of idle dynamic nodes. This allows the group of idle dynamic nodes to obtain the image of the business application from the distributed file system when creating a copy of the business application, even if the image of the business application is not stored locally. The image is used to create the copy of the business application.
[0184] In some embodiments of this disclosure, the device may further include:
[0185] The detection module is used to detect whether the amount of mirrored data in the distributed file system has reached a threshold.
[0186] If the detection result of the detection module is yes, then delete the unused images in the distributed file system and the images whose last time of use is longer than the duration threshold.
[0187] In some embodiments of this disclosure, the node set further includes idle fixed nodes, used fixed nodes, and used dynamic nodes. No business applications are running within the idle fixed nodes, while business applications are running within the used fixed nodes and used dynamic nodes. The price of the used fixed nodes is lower than the price of the used dynamic nodes, and the price of the idle fixed nodes is lower than the price of the idle dynamic nodes. The device may also include:
[0188] The creation module is used to determine whether the remaining memory of a node is greater than or equal to the rated memory required for the replica when it is determined that a replica needs to be created for the first time for a business application. It follows the order from used fixed nodes, idle fixed nodes, used dynamic nodes to idle dynamic nodes, and so on. When it is determined that the remaining memory of any node is greater than or equal to the rated memory required for the replica, a replica is created in that node.
[0189] In some embodiments of this disclosure, the node set further includes used dynamic nodes, in which business applications are running; the apparatus may also include:
[0190] The judgment module is used to determine whether the used dynamic nodes meet the first configuration conditions for each replica that needs to be expanded when it is determined that a currently running business application needs to be expanded. The first configuration conditions include: the remaining memory of the node is greater than or equal to the rated memory required by the replica, and the node's configured dependency environment includes the dependency environment required by the replica.
[0191] Create a module to create a copy in the used dynamic node if the judgment result of the judgment module is satisfied;
[0192] The judgment module is also used to determine whether there is a first idle dynamic node that meets the first configuration condition;
[0193] The creation module is also used to create a copy of the first idle dynamic node if the judgment result of the judgment module is the first idle dynamic node.
[0194] In some embodiments of this disclosure, the device may further include:
[0195] The judgment module is further configured to, after determining whether there is a first idle dynamic node that meets the first configuration condition, if there is no first idle dynamic node, determine whether the used dynamic node meets the second configuration condition; wherein the second configuration condition includes: the remaining memory of the node is greater than or equal to the rated memory required by the replica;
[0196] The module creation function is also used to create a copy of the module in the used dynamic node if the judgment result of the module is satisfied.
[0197] The judgment module is also used to determine whether there is a second idle dynamic node that meets the second configuration condition if the condition is not met.
[0198] The creation module is also used to create a copy of the second idle dynamic node if the judgment result of the judgment module is the second idle dynamic node.
[0199] In some embodiments of this disclosure, the device may further include:
[0200] The deletion module is used to delete a specified number of replicas of a currently running business application from each specified replica of that business application when it is determined that the application needs to be scaled down. The dependency environment configured on the node where the specified replicas are located is different from the dependency environment required by the replicas of that business application.
[0201] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0202] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0203] like Figure 10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the electronic device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0204] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of displays, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0205] The computing unit 1001 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as resource management methods. For example, in some embodiments, the resource management method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the resource management method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform resource management methods by any other suitable means (e.g., by means of firmware).
[0206] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0207] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0208] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0209] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0210] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0211] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0212] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0213] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A resource management method, comprising: Determine whether the node set needs to be expanded or reduced; If it is determined that the node set needs to be expanded, nodes are selected from the node inventory in order of price from low to high. The selected nodes are added to the node set as idle dynamic nodes until the number of idle dynamic nodes in the node set is a preset number. In this case, no business applications are running in the nodes in the node inventory or in the idle dynamic nodes. If it is determined that the node set needs to be scaled down, then nodes are deleted from the idle dynamic nodes of the node set in descending order of price until the number of idle dynamic nodes in the node set is the preset number. The node set also includes used dynamic nodes, and the used dynamic nodes are running business applications. Determine whether there is a target node among the idle dynamic nodes added to the node set during this node expansion, and the price of the target node is lower than the price of the used dynamic nodes; If it exists, then migrate the business application copy in the used dynamic node to the target node; Update the used dynamic nodes to idle dynamic nodes, and update the target node to a used dynamic node.
2. The method according to claim 1, wherein the node set further includes used fixed nodes and idle fixed nodes, wherein the used fixed nodes are running business applications, the idle fixed nodes are not running business applications, the price of the used fixed nodes is lower than the price of the used dynamic nodes, and the price of the idle fixed nodes is lower than the price of the idle dynamic nodes. Before determining whether the target node exists among the idle dynamic nodes added to the node set during this node expansion, the method further includes: Determine whether a target replica is running in the used dynamic node, wherein the target replica is the first replica created for the currently running business application; If so, determine whether the remaining memory of the used fixed node is greater than or equal to the rated memory required by the target replica; If it is greater than or equal to, then the target copy is migrated to the used fixed node; If the value is less than the target replica, the target replica will be migrated to an idle fixed node, and the idle fixed node will be updated to a used fixed node.
3. The method according to claim 1, after selecting the nodes as idle dynamic nodes and adding them to the node set until the number of idle dynamic nodes in the node set reaches a preset number, the method further includes: Based on the configuration volume of various preset dependent environment types counted within the current statistical period, the first preset number of target types are selected in descending order of configuration volume. For each target type, based on the configuration volume of each dependent environment of that target type in the current statistical period, the second preset number of dependent environments are selected in descending order of configuration volume. Based on the proportion of configuration amounts of various target types within the current statistical period, the idle dynamic nodes added to the node set during this node expansion are grouped. For each group of idle dynamic nodes, configure a second preset number of dependency environments of a specified type within that group of idle dynamic nodes, wherein the specified type is the target type corresponding to the proportion item to which the group of idle dynamic nodes belongs.
4. The method according to claim 3, after configuring a second preset number of dependency environments of a specified type within each group of idle dynamic nodes, the method further includes: For each group of idle dynamic nodes, the image storage path of the group of idle dynamic nodes is mounted to the distributed file system. This allows the group of idle dynamic nodes to obtain the image of the business application from the distributed file system when creating a copy of the business application, even if the image of the business application is not stored locally. The image is used to create the copy of the business application.
5. The method according to claim 4, further comprising: Detect whether the amount of mirrored data in the distributed file system has reached a threshold; If so, delete the unused images and images in the distributed file system whose last usage time is longer than the duration threshold.
6. The method according to claim 1, wherein the node set further comprises idle fixed nodes, used fixed nodes, and used dynamic nodes, wherein no business applications are running in the idle fixed nodes, and business applications are running in the used fixed nodes and the used dynamic nodes, wherein the price of the used fixed nodes is lower than the price of the used dynamic nodes, and the price of the idle fixed nodes is lower than the price of the idle dynamic nodes; the method further comprises: If it is determined that a replica needs to be created for the first time for a business application, the remaining memory of each node is determined sequentially in the order of the used fixed node, the idle fixed node, the used dynamic node and the idle dynamic node, to see if it is greater than or equal to the rated memory required by the replica, until it is determined that the remaining memory of any node is greater than or equal to the rated memory required by the replica, and then the replica is created in that node.
7. The method according to claim 1, wherein the node set further includes used dynamic nodes, and the used dynamic nodes are running business applications; the method further includes: If it is determined that a currently running business application needs to be expanded, for each replica that needs to be expanded from the business application, it is determined whether the used dynamic node meets the first configuration condition; wherein, the first configuration condition includes: the remaining memory of the node is greater than or equal to the rated memory required by the replica, and the node's configured dependency environment includes the dependency environment required by the replica; If the conditions are met, then create the copy in the used dynamic node; If not, determine whether there is a first idle dynamic node that meets the first configuration condition; If the first idle dynamic node exists, then create the replica in the first idle dynamic node.
8. The method according to claim 7, after determining whether there is a first idle dynamic node satisfying the first configuration condition, the method further includes: If the first idle dynamic node does not exist, then determine whether the used dynamic node meets the second configuration condition; wherein, the second configuration condition includes: the remaining memory of the node is greater than or equal to the rated memory required by the replica; If the conditions are met, then create the copy in the used dynamic node; If not, determine whether there is a second idle dynamic node that meets the second configuration condition; If a second idle dynamic node exists, then create the copy in the second idle dynamic node.
9. The method according to any one of claims 1-8, further comprising: If it is determined that a currently running business application needs to be scaled down, a specified number of replicas of the business application are deleted from each specified replica. The dependency environment configured on the node where the specified replicas are located is different from the dependency environment required by the replicas of the business application.
10. A resource management device, comprising: The determination module is used to determine whether the node set needs to be expanded or reduced in size. The node expansion module is used to select nodes from the node inventory in order of price from low to high if the determining module determines that the node set needs to be expanded. The selected nodes are added to the node set as idle dynamic nodes until the number of idle dynamic nodes in the node set is a preset number. In this case, no business applications are running in the nodes in the node inventory or in the idle dynamic nodes. The node scaling-down module is used to delete nodes from the idle dynamic nodes of the node set in descending order of price if the determining module determines that the node set needs to be scaled down, until the number of idle dynamic nodes in the node set is the preset number. The node set also includes used dynamic nodes, and the used dynamic nodes are running business applications. The judgment module is used to determine whether there is a target node among the idle dynamic nodes added to the node set during this node expansion, and the price of the target node is lower than the price of the used dynamic nodes; The migration module is used to migrate the business application copy in the used dynamic node to the target node if the judgment result of the judgment module is that it exists. The update module is used to update the used dynamic nodes to idle dynamic nodes and the target node to a used dynamic node.
11. The apparatus according to claim 10, wherein the node set further comprises used fixed nodes and idle fixed nodes, wherein the used fixed nodes are running business applications, the idle fixed nodes are not running business applications, the price of the used fixed nodes is lower than the price of the used dynamic nodes, and the price of the idle fixed nodes is lower than the price of the idle dynamic nodes; the apparatus further comprises: The judgment module is also used to determine whether a target replica is running in the used dynamic nodes before determining whether a target node exists in the idle dynamic nodes added to the node set during this node expansion. The target replica is a replica created for the first time for the currently running business application. The judgment module is further configured to determine whether the remaining memory of the used fixed node is greater than or equal to the rated memory required by the target replica if the condition is met. The migration module is further configured to migrate the target copy to the used fixed node if the judgment result of the judgment module is greater than or equal to the target copy. The migration module is further configured to migrate the target copy to an idle fixed node and update the idle fixed node to a used fixed node if the judgment result of the judgment module is less than 1.
12. The apparatus of claim 10, further comprising: The filtering module is used to filter a first preset number of target types according to the configuration volume of various preset dependent environment types counted in the current statistical period, in descending order of configuration volume, after the selected nodes are added as idle dynamic nodes to the node set until the number of idle dynamic nodes in the node set is a preset number. The filtering module is also used to filter a second preset number of dependent environments for each target type, based on the configuration quantity of each dependent environment of that target type as statistically recorded in the current statistical period, in descending order of configuration quantity. The grouping module is used to group the idle dynamic nodes added to the node set during this node expansion according to the proportion of the configuration amount of various target types in the current statistical period. The configuration module is used to configure a second preset number of dependency environments of a specified type within each group of idle dynamic nodes, wherein the specified type is the target type corresponding to the proportion item to which the group of idle dynamic nodes belongs.
13. The apparatus of claim 12, further comprising: The mounting module is used to mount the image storage path of each group of idle dynamic nodes to a distributed file system after configuring a second preset number of dependency environments of a specified type within each group of idle dynamic nodes. This allows the group of idle dynamic nodes to obtain the image of the business application from the distributed file system when creating a copy of the business application, even if the image of the business application is not stored locally. The image is used to create the copy of the business application.
14. The apparatus of claim 13, further comprising: The detection module is used to detect whether the amount of mirrored data in the distributed file system has reached a threshold. If the detection result of the detection module is yes, then the unused images and images in the distributed file system that were last used more than a duration threshold from the current time will be deleted.
15. The apparatus according to claim 10, wherein the node set further comprises idle fixed nodes, used fixed nodes, and used dynamic nodes, wherein no business applications are running in the idle fixed nodes, and business applications are running in the used fixed nodes and the used dynamic nodes, wherein the price of the used fixed nodes is lower than the price of the used dynamic nodes, and the price of the idle fixed nodes is lower than the price of the idle dynamic nodes; the apparatus further comprises: A creation module is used to, when it is determined that a replica needs to be created for the first time for a business application, sequentially determine whether the remaining memory of a node is greater than or equal to the rated memory required by the replica, in the order of the used fixed node, the idle fixed node, the used dynamic node to the idle dynamic node, until it is determined that the remaining memory of any node is greater than or equal to the rated memory required by the replica, and then create the replica in that node.
16. The apparatus according to claim 10, wherein the node set further includes used dynamic nodes, wherein a business application is running in the used dynamic nodes; the apparatus further includes: The judgment module is used to determine whether the used dynamic nodes meet the first configuration conditions for each replica that needs to be expanded for a currently running business application when it is determined that the business application needs to be expanded. The first configuration conditions include: the remaining memory of the node is greater than or equal to the rated memory required by the replica, and the dependency environment configured by the node includes the dependency environment required by the replica. A creation module is used to create the copy in the used dynamic node if the judgment result of the judgment module is satisfied. The judgment module is also used to determine whether there is a first idle dynamic node that meets the first configuration condition; The creation module is further configured to create the copy in the first idle dynamic node if the judgment result of the judgment module is the first idle dynamic node.
17. The apparatus of claim 16, further comprising: The determination module is further configured to, after determining whether there is a first idle dynamic node that satisfies the first configuration condition, if there is no first idle dynamic node, determine whether the used dynamic node satisfies the second configuration condition; wherein the second configuration condition includes: the remaining memory of the node is greater than or equal to the rated memory required by the replica; The creation module is further configured to create the copy in the used dynamic node if the judgment result of the judgment module is satisfied. The judgment module is further configured to determine whether there is a second idle dynamic node that satisfies the second configuration condition if the condition is not met. The creation module is further configured to create the copy in the second idle dynamic node if the judgment result of the judgment module is the second idle dynamic node.
18. The apparatus according to any one of claims 10-17, further comprising: The deletion module is used to delete a specified number of replicas of a currently running business application from each specified replica of that business application when it is determined that the application needs to be scaled down. The dependency environment configured on the node where the specified replicas are located is different from the dependency environment required by the replicas of the business application.
19. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.
21. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.
Citation Information
Patent Citations
Method, apparatus, device, and storage medium for scheduling jobs in cluster
CN109117265A
Resource adjustment method and device, electronic equipment and computer readable storage medium
CN114518957A