Cluster node computing power estimation method and device and cloud management platform equipment
Patent Information
- Application Number
- CN202510697804.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-28
- Publication Date
- 2025-07-04
Smart Images
Figure CN120263673A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of cloud computing, and particularly to a method and device for estimating the computing power of cluster nodes and a cloud management platform device. Background Art
[0002] With the development of cloud computing technology, Kubernetes (abbreviated as k8s, a container orchestration platform) has become the first choice for enterprises and developers to build and manage containerized applications. Kubernetes has the ability to automate the deployment, scaling, and management of containerized applications, making the deployment and operation and maintenance of application programs simpler and more efficient. In this context, as a tool for centrally managing multiple Kubernetes clusters, the cloud management platform (CMP, Cloud Management Platform) has also received increasing attention. The cloud management platform can help enterprises achieve unified management across cloud environments, improve resource utilization, reduce operation and maintenance costs, and at the same time provide security and compliance guarantees.
[0003] In a cloud computing cluster (such as a Kubernetes cluster) managed by a cloud management platform, a node is a basic unit of a workload. A node can be a physical machine or a virtual machine, and contains a set of resources (such as a central processing unit CPU, memory, and storage, etc.). Accurately estimating the computing power of a node can assist cluster administrators in resource scheduling, load balancing, and cost control. However, due to the hardware differences of the nodes themselves, the complexity of the operating environment, and the dynamic changes of the workload, it is challenging to achieve accurate computing power estimation.
[0004] In related technologies, the computing power estimation methods for cluster nodes often have problems with insufficient accuracy. For example, in some solutions, only a single hardware metric (such as the number of CPU cores or the memory capacity) is selected to approximately estimate the computing power of the node. For example, simply judging the computing power of a node based on the number of CPU cores, regarding each core as having the same computing ability, and simply multiplying the number of cores by a fixed coefficient to obtain the computing power value. Therefore, how to improve the accuracy of computing power estimation for cluster nodes managed by a cloud management platform and provide a reliable basis for resource allocation and scheduling of the cloud management platform is an urgent problem to be solved today. Summary of the Invention
[0005] The purpose of the present invention is to provide a method and device for estimating the computing power of cluster nodes and a cloud management platform device, so as to improve the accuracy of computing power estimation for cluster nodes managed by a cloud management platform and provide a reliable basis for resource allocation and scheduling of the cloud management platform.
[0006] To solve the above technical problems, the present invention provides a method for estimating the computing power of cluster nodes, which is applied to a cloud management platform device and includes:
[0007] Send a preset task to each target cluster node; wherein, the preset task is used to execute a preset hardware parameter collection script at startup;
[0008] Obtain the hardware parameter information of each of the target cluster nodes; wherein, the hardware parameter information is the information collected by the target cluster node when executing the preset hardware parameter collection script, and the hardware parameter information includes at least two of central processor performance information, memory performance information, and storage performance information;
[0009] Calculate the pre-estimated computing power of each of the target cluster nodes according to the hardware parameter information.
[0010] On the other hand, the central processor performance information includes at least one of the number of central processors, maximum frequency, base frequency, number of cores, and number of threads per core; the memory performance information includes the memory capacity and / or memory frequency; the storage performance information includes at least one of storage capacity, read rate, write rate, and storage type.
[0011] On the other hand, the hardware parameter information includes the central processor performance information, the memory performance information, and the storage performance information. Calculating the pre-estimated computing power of each of the target cluster nodes according to the hardware parameter information includes:
[0012] Calculate the central processor performance value C, memory performance value M, and storage performance value D of the current cluster node according to the hardware parameter information of the current cluster node; wherein, the current cluster node is any one of the target cluster nodes; C = num_chips × (w freq1 × (max_freq + base_freq) / 2 + w cores × cores × threads_per_core), M = w volume1 × memory_volume + w freq2 × memory_freq, D = w volume2 × storage_volume + w speed × read_speed + w speed × write_speed; num_chips is the number of central processors, max_freq is the maximum frequency, base_freq is the base frequency, cores is the number of cores, threads_per_core is the number of threads per core, w freq1 is the first preset frequency weight, w cores is the preset core weight; memory_volume is the memory capacity, memory_freq is the memory frequency, w volume1 is the first preset capacity weight, w freq2is the second preset frequency weight; storage_volume is the storage capacity, read_speed is the read rate, write_speed is the write rate, w volume2 is the second preset capacity weight, w speed is the preset rate weight;
[0013] Through w C ×C + w M ×M + w D ×D, calculate the pre - estimated force of the current cluster node; where, w C is the preset processor weight, w M is the preset memory weight, w D is the preset storage weight, w C +w M +w D = 1, w C > w M > w D .
[0014] On the other hand, according to the hardware parameter information of the current cluster node, calculate the central processor performance value C, memory performance value M and storage performance value D of the current cluster node, including:
[0015] If the storage type in the memory performance information of the current cluster node is a solid - state drive, then make w speed Adopt the preset solid - state drive weight value; if the storage type in the memory performance information of the current cluster node is not a solid - state drive, then make w speed Adopt the preset other hard - drive weight value; where, the preset solid - state drive weight value is greater than the preset other hard - drive weight value.
[0016] On the other hand, the preset task includes mount directory information, and the mount directory information includes at least one of the boot program directory, operating system detailed information directory, system directory and system - level library file storage directory.
[0017] On the other hand, obtain the hardware parameter information of each of the target cluster nodes, including:
[0018] Monitor the logs generated by each of the target cluster nodes during the execution of the preset task, and obtain the hardware parameter information of each of the target cluster nodes.
[0019] On the other hand, the target cluster node is a node in the container orchestration platform cluster.
[0020] On the other hand, according to the hardware parameter information, calculate the pre - estimated force of each of the target cluster nodes, including:
[0021] Configure the characteristic tags corresponding to each of the target cluster nodes according to the hardware parameter information;
[0022] Calculate the pre-estimated computing power of each of the target cluster nodes according to the values of the characteristic tags.
[0023] The present invention also provides a computing power estimation device for cluster nodes, which is applied to a cloud management platform device and includes:
[0024] A task distribution module, configured to distribute a preset task to each target cluster node; wherein, the preset task is used to execute a preset hardware parameter collection script when starting up;
[0025] A parameter acquisition module, configured to acquire the hardware parameter information of each of the target cluster nodes; wherein, the hardware parameter information is the information collected by the target cluster node when executing the preset hardware parameter collection script, and the hardware parameter information includes at least two of central processor performance information, memory performance information, and storage performance information;
[0026] A computing power estimation module, configured to calculate the pre-estimated computing power of each of the target cluster nodes according to the hardware parameter information.
[0027] In addition, the present invention also provides a cloud management platform device, including:
[0028] A memory, configured to store a computer program;
[0029] A processor, configured to implement the steps of the computing power estimation method for cluster nodes as described above when executing the computer program.
[0030] A computing power estimation method for cluster nodes provided by the present invention, which is applied to a cloud management platform device and includes: distributing a preset task to each target cluster node; wherein, the preset task is used to execute a preset hardware parameter collection script when starting up; acquiring the hardware parameter information of each target cluster node; wherein, the hardware parameter information is the information collected by the target cluster node when executing the preset hardware parameter collection script, and the hardware parameter information includes at least two of central processor performance information, memory performance information, and storage performance information; calculating the pre-estimated computing power of each target cluster node according to the hardware parameter information.
[0031] It can be seen that by sending preset tasks to each target cluster node, the present invention enables the target cluster node to execute a preset hardware parameter collection script when starting the preset task, and penetratively collect the underlying hardware parameter information; by calculating the pre-estimated computing power of each target cluster node according to the hardware parameter information, it can comprehensively consider various hardware resources and their synergistic effects, significantly improve the accuracy of computing power estimation of the cluster nodes managed by the cloud management platform, and provide a reliable basis for resource allocation and scheduling of the cloud management platform. In addition, the present invention also provides a computing power estimation device for a cluster node and a cloud management platform device, which also have the above beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.
[0033] Figure 1 It is a flowchart of a method for estimating the computing power of a cluster node provided by an embodiment of the present invention;
[0034] Figure 2 It is a schematic diagram of the deployment architecture of another method for estimating the computing power of a cluster node provided by an embodiment of the present invention;
[0035] Figure 3 It is a schematic diagram of the content of a preset hardware parameter collection script provided by an embodiment of the present invention;
[0036] Figure 4 It is a structural block diagram of a computing power estimation device for a cluster node provided by an embodiment of the present invention;
[0037] Figure 5 It is a schematic diagram of the simple structure of a cloud management platform device provided by an embodiment of the present invention;
[0038] Figure 6 It is a schematic diagram of the specific structure of a cloud management platform device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.
[0040] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for estimating the computing power of cluster nodes provided by an embodiment of the present invention. This method is applied to cloud management platform devices and may include:
[0041] Step 101: Send a preset task to each target cluster node; wherein, the preset task is used to execute a preset hardware parameter collection script when starting up.
[0042] It can be understood that the cloud management platform device provided in this embodiment can be a device running the cloud management platform, such as a server. The target cluster nodes in this embodiment can be the nodes (Nodes) in a cloud computing cluster (such as a Kubernetes cluster) managed by the cloud management platform device that need to estimate computing power.
[0043] Correspondingly, for the specific quantity and type of the target cluster nodes in this embodiment, they can be set by designers according to practical scenarios and user requirements. For example, the target cluster nodes include all or part of the nodes in the Kubernetes cluster managed by the cloud management platform device; they can also include all or part of the nodes in other cloud computing clusters managed by the cloud management platform device. This embodiment does not make any restrictions on this.
[0044] Accordingly, the preset task in this embodiment can be a task pre-set for executing a preset hardware parameter collection script when starting up, that is, the target cluster node can start running the preset task sent by the cloud management platform device, execute the preset hardware parameter collection script during the running of the preset task, and collect detailed hardware parameter information inside the node. For example, when the target cluster node is a node in the Kubernetes cluster managed by the cloud management platform device, the preset task can be a job (task) sent by the cloud management platform device for executing a series of shell (a programming language) scripts (i.e., the preset hardware parameter collection script), and these scripts can deeply obtain detailed hardware parameter information of the Kubernetes node.
[0045] Furthermore, the preset task in this embodiment can include mount directory information, and the mount directory information includes at least one of the bootloader (such as / boot) directory, the operating system detailed information (such as / etc / os-release) directory, the system (such as / sys) directory, and the system-level library file storage (such as / usr / lib) directory. Based on the hardware detection method of mounting specific directories, by customizing the preset hardware parameter collection script (such as a Shell script) to mount the system directory (i.e., the directory of the mount directory information), directly read the kernel-level hardware parameters, and implement a penetration hardware detection technology.
[0046] Among them, for the specific time when the cloud management platform device in this embodiment issues a preset task to each target cluster node, it can be set by the designer according to the practical scenario and user requirements. For example, when the cloud management platform device takes over a cloud computing cluster, all nodes in the cloud computing cluster can be determined as target cluster nodes, and a preset task can be automatically issued to all nodes in the cloud computing cluster. For example, after the cloud management platform device takes over a Kubernetes cluster, it can automatically issue a job (i.e., a preset task) to each Node in the Kubernetes cluster. In order to obtain the hardware changes of the cluster nodes in real time, in some embodiments, the cloud management platform device can also issue a preset task to each target cluster node at a preset time interval, or control each target cluster node to start and run a preset task at a preset time interval, so that each target cluster node can periodically run the preset task. In other embodiments, the cloud management platform device can also use the obtained computing power estimation instruction triggered by the user, use the node corresponding to the computing power estimation instruction as the target cluster node, and issue a preset task to the target cluster node to implement the preset task issuance triggered manually by the user. This embodiment does not impose any restrictions on this.
[0047] Step 102: Obtain the hardware parameter information of each target cluster node; among them, the hardware parameter information is the information collected by the target cluster node when executing a preset hardware parameter collection script, and the hardware parameter information includes at least two of the central processor performance information, memory performance information, and storage performance information.
[0048] It can be understood that the hardware parameter information in this embodiment can be the information collected by each target cluster node when executing a preset hardware parameter collection script during the process of running the preset task issued by the cloud management platform device. That is, after the target cluster node starts the preset task, it will execute the preset hardware parameter collection script to detect the hardware parameter information, such as the central processor (CPU) performance information, memory performance information, and storage performance information, etc.
[0049] Correspondingly, for the specific quantity and content of the hardware parameter information of the target cluster node in this embodiment, it can be set by the designer according to the practical scenario and user requirements. For example, the hardware parameter information can include at least two of the central processor performance information, memory performance information, and storage performance information; for example, the hardware parameter information can include the central processor performance information, memory performance information, and storage performance information. The hardware parameter information can also include other information such as network performance information. For example, the hardware parameter information can include the central processor performance information, memory performance information, storage performance information, and network performance information. This embodiment does not impose any restrictions on this.
[0050] Correspondingly, the above-mentioned central processing unit performance information may include at least one of the number of central processing units (num_chips), maximum frequency (max_freq), base frequency (base_freq), number of cores (cores), and number of threads per core (threads_per_core). For example, the central processing unit performance information may include the number of central processing units, maximum frequency, base frequency, number of cores, and number of threads per core. The above-mentioned memory performance information may include memory capacity (memory_volume) and / or memory frequency (memory_freq). For example, the memory performance information may include memory capacity and memory frequency. The above-mentioned storage performance information includes at least one of storage capacity (storage_volume), read speed (read_speed), write speed (write_speed), and storage type. For example, the storage performance information may include storage capacity, read speed, write speed, and storage type.
[0051] It should be noted that for the specific method of the cloud management platform device to obtain the hardware parameter information of each target cluster node in this step, it can be set by the designer according to the practical scenario and user requirements. For example, the cloud management platform device monitors the logs generated by each target cluster node during the execution of the preset task to obtain the hardware parameter information of each target cluster node. That is to say, when the target cluster node executes the preset hardware parameter collection script during the execution of the preset task, it will detect the hardware parameter information and print out the relevant logs, so that the cloud management platform device can continuously monitor these logs and extract the information related to the hardware characteristics of the target cluster node (i.e., hardware parameter information). As Figure 2 shown, after the node in the Kubernetes cluster starts the job (i.e., the preset task) issued by the cloud management platform device (cloud management platform), it executes the shell script (i.e., the preset hardware parameter collection script), detects the hardware parameter information such as the central processing unit performance information, memory performance information, and storage performance information, and prints out the relevant logs. The cloud management platform device can use the data acquisition module to continuously monitor the logs generated by the job after issuing the job and extract the hardware parameter information. The cloud management platform device can also directly receive the hardware parameter information returned by each target cluster node after the execution of the preset task. This embodiment does not make any restrictions on this.
[0052] For example, the content of the above shell script can be as Figure 3As shown, through commands of the Linux (an operating system) system, such as dmidecode (a command-line tool for extracting hardware information in the Linux system), free (a command for displaying the memory usage of the current system), and lsblk (list block devices, a command-line tool for listing information of block devices), etc., the hardware parameter information of the node is obtained, and relevant logs are printed.
[0053] Step 103: Calculate the pre-estimated force of each target cluster node according to the hardware parameter information.
[0054] It can be understood that in this step, the cloud management platform device can calculate the pre-estimated force of each target cluster node according to the obtained hardware parameter information of each target cluster node, so as to comprehensively consider multiple hardware resources (such as at least two of the central processor performance information, memory performance information, and storage performance information) and their synergistic effects, significantly improve the accuracy of the computing power estimation of the cluster nodes managed by the cloud management platform, and provide a reliable basis for the resource allocation and scheduling of the cloud management platform.
[0055] Correspondingly, for the specific method of calculating the pre-estimated force of each target cluster node by the cloud management platform device according to the hardware parameter information in this step, it can be set by the designer according to the practical scenario and user requirements. For example, when the hardware parameter information includes central processor performance information, memory performance information, and storage performance information, the central processor performance information includes the number of central processors, maximum frequency, base frequency, number of cores, and number of threads per core, the memory performance information includes memory capacity and memory frequency, and the storage performance information includes storage capacity, read rate, and write rate, the cloud management platform device can calculate the central processor performance value C, memory performance value M, and storage performance value D of the current cluster node according to the hardware parameter information of the current cluster node; through w C ×C + w M ×M + w D ×D, calculate the pre-estimated force of the current cluster node; where the current cluster node is any target cluster node; C = num_chips × (w freq1 ×(max_freq + base_freq) / 2 + w cores ×cores × threads_per_core), M = w volume1 ×memory_volume + w freq2 ×memory_freq, D = w volume2 ×storage_volume + w speed ×read_speed + w speed× write_speed; num_chips is the number of central processing units, max_freq is the maximum frequency, base_freq is the base frequency, cores is the number of cores, threads_per_core is the number of threads per core, w freq1 is the first preset frequency weight (such as 0.2), w cores is the preset core weight (such as 0.5); memory_volume is the memory capacity, memory_freq is the memory frequency, w volume1 is the first preset capacity weight (such as 0.6), w freq2 is the second preset frequency weight (such as 0.4); storage_volume is the storage capacity, read_speed is the read rate, write_speed is the write rate, w volume2 is the second preset capacity weight (such as 0.6), w speed is the preset rate weight; w C is the preset processor weight, w M is the preset memory weight, w D is the preset storage weight, w C + w M + w D = 1, w C > w M > w D , such as w C = 0.5, w M = 0.3, w D = 0.2.
[0056] Correspondingly, the preset rate weight (w speed ) can be a preset fixed value; when the storage performance information further includes the storage type, w speed= if SSD then w SSD else w HDD , w SSD is the preset solid-state drive weight value, w HDD is the preset other hard drive weight value. That is to say, the process of calculating the central processing unit performance value C, memory performance value M, and storage performance value D of the current cluster node according to the hardware parameter information of the current cluster node can include: if the storage type in the memory performance information of the current cluster node is a solid-state drive, then make w speed adopt the preset solid-state drive weight value; if the storage type in the memory performance information of the current cluster node is not a solid-state drive, then make w speed adopt the preset other hard drive weight value; where the preset solid-state drive weight value is greater than the preset other hard drive weight value.
[0057] In some other embodiments, the hardware parameter information may include central processor performance information and memory performance information. In this step, the cloud management platform device may calculate the central processor performance value C and the memory performance value M of the current cluster node according to the hardware parameter information of the current cluster node; through w C1 ×C + w M1 ×M, calculate the pre-estimated force of the current cluster node; where the current cluster node is any target cluster node; w C1 + w M1 = 1, w C1 >w M1 ; w C1 is the first preset processor weight, and w M1 is the first memory weight. The hardware parameter information may also include central processor performance information, memory performance information, storage performance information, and network performance information. In this step, the cloud management platform device may calculate the central processor performance value C, the memory performance value M, the storage performance value D, and the network performance value N of the current cluster node according to the hardware parameter information of the current cluster node; through w C2 ×C + w M2 ×M + w D1 ×D + w N ×N, calculate the pre-estimated force of the current cluster node; where the current cluster node is any target cluster node; w C2 + w M2 + w D1 + w N = 1, w C2 >w M2 >w D1 >w N ; w C2 is the second preset processor weight, w M2 is the second memory weight, w D1 is the first preset storage weight, and w N is the preset network weight.
[0058] Furthermore, in order to facilitate the display and update of the hardware parameter information of each target cluster node, in some embodiments, step 102 may include: configuring feature tags corresponding to each target cluster node according to the hardware parameter information; calculating the pre-estimated force of each target cluster node according to the values of the feature tags. Such as Figure 2As shown, after the data acquisition module extracts the hardware parameter information, it can call the feature label update module. Using the feature label update module, the hardware parameter information is updated to be reflected in the values of the feature labels corresponding to each target cluster node. For example, cpu.node / count (CPU count label) = 2 and disk.node / capacity (storage memory label) = 10240, etc.; the cloud management platform device can use the computing power module to continuously monitor the labels of each target cluster node. When the feature labels in the label of a Node change, the computing power calculation can be triggered; relying on the values of the feature labels and the formula for computing power calculation, the pre-estimated computing power of the node can be calculated; the pre-estimated computing power can also be updated to the Node in the form of a label, such as capacity.node / score (computing power label) = 1600; that is, the label of the target cluster node can include feature labels and computing power labels; the cloud management platform device can also send the feature labels and / or computing power labels to the corresponding target cluster nodes to be directly integrated into the Kubernetes native label system, enabling the cloud management platform device to quickly screen high-computing-power nodes for critical task scheduling and improving the resource allocation decision-making speed by 35%.
[0059] Correspondingly, the process of calculating the pre-estimated computing power of each target cluster node according to the values of the feature labels can be set in a similar manner to the process of directly calculating the pre-estimated computing power of each target cluster node according to the hardware parameter information above. For example, when the hardware parameter information includes central processor performance information, memory performance information, and storage performance information, the cloud management platform device can calculate the central processor performance value C, memory performance value M, and storage performance value D of the current cluster node according to the values of the feature labels corresponding to the current cluster node; through w C ×C + w M ×M + w D ×D, calculate the pre-estimated computing power of the current cluster node; where the current cluster node is any target cluster node; C = num_chips×(w freq1 ×(max_freq + base_freq) / 2 + w cores ×cores×threads_per_core), M = w volume1 ×memory_volume + w freq2 ×memory_freq, D = w volume2 ×storage_volume + w speed ×read_speed + w speed× write_speed; num_chips is the number of central processing units (i.e., the value of the CPU number tag), max_freq is the maximum frequency (i.e., the value of the maximum frequency tag), base_freq is the base frequency (i.e., the value of the base frequency tag), cores is the number of cores (i.e., the value of the number of cores tag), threads_per_core is the number of threads per core (i.e., the value of the number of threads per core tag), w freq1 is the first preset frequency weight (e.g., 0.2), w cores is the preset core weight (e.g., 0.5); memory_volume is the memory capacity (i.e., the value of the memory capacity tag), memory_freq is the memory frequency (i.e., the value of the memory frequency tag), w volume1 is the first preset capacity weight (e.g., 0.6), w freq2 is the second preset frequency weight (e.g., 0.4); storage_volume is the storage capacity (i.e., the value of the storage capacity tag), read_speed is the read rate (i.e., the value of the read rate tag), write_speed is the write rate (i.e., the value of the write rate tag), w volume2 is the second preset capacity weight (e.g., 0.6), w speed is the preset rate weight; w C is the preset processor weight, w M is the preset memory weight, w D is the preset storage weight, w C + w M + w D = 1, w C > w M > w D , e.g., w C = 0.5, w M = 0.3, w D = 0.2; w speed= if SSD then w SSD else w HDD .
[0060] For example, when a user uses a cloud management platform device to manage a k8s cluster, assume that the k8s cluster has three worker nodes: node1, node2, and node3; the cloud management platform device can send jobs (i.e., preset tasks) to node1, node2, and node3; after the job is started, a shell script will be executed. Assume that the following data is detected on node1: CPU: the number is 2, with a total of 24 cores, the number of threads per core is 2, the base frequency is 2200 MHz, and the maximum frequency is 3200 MHz; Memory: the memory capacity is 128G, and the frequency is 2933 MHz; Storage: a common mechanical hard disk, with a storage capacity of 10240G, and the read and write speeds are both 100M / s. The cloud management platform device can label node1 with the following characteristic tags according to the detected hardware parameter information: cpu.node / count (CPU count tag) = 2; cpu.node / cores (core count tag) = 24; cpu.node / threads.per.core (threads per core tag) = 2; cpu.node / mhz.base (base frequency tag) = 2200; cpu.node / mhz.max (maximum frequency tag) = 3200; memory.node / capacity (memory capacity tag) = 128; memory.node / mhz (memory frequency tag) = 2933; disk.node / capacity (storage capacity tag) = 10240; disk.node / speed (read and write rate tag) = 100; According to the values of the characteristic tags, the computing power value of node1 can be calculated as 2187.8 using the above computing power estimation formula; a computing power tag can be labeled for node1: capacity.node / score = 2187.8.
[0061] Correspondingly, this embodiment does not limit the specific naming rules of the labels (such as characteristic labels and computing power labels) corresponding to each target cluster node. For example, the characteristic labels can be named in the way of <resource type>.ndoe / <parameter>, or <resource type>.<node identifier> / <parameter> to facilitate the user's viewing.
[0062] Furthermore, the method provided in this embodiment can also include the application process of the pre-estimated computing power of each target cluster node. For example, when the target cluster node is a node in a Kubernetes cluster, the cloud management platform device can allocate newly created containers (Pods) according to the pre-estimated computing power of each target cluster node; for example, the newly created containers are preferentially allocated to target cluster nodes with higher pre-estimated computing power; for example, when the remaining resources of a node with higher pre-estimated computing power are sufficient, it is preferentially expanded to such nodes to avoid overloading low-computing-power nodes, so as to achieve intelligent scaling management of the Kubernetes cluster.
[0063] In some other embodiments, the cloud management platform device may, according to the pre-estimated computing power of each target cluster node and the read / write rate of the storage, schedule disk-intensive tasks to the target cluster nodes with higher pre-estimated computing power and high throughput, so as to reduce latency and improve throughput, and optimize load balancing. The cloud management platform device may also classify the target cluster nodes according to the pre-estimated computing power of the target cluster nodes, determine the types of each target cluster node (such as CPU-intensive or memory-intensive); and allocate the tasks to be processed to the corresponding target cluster nodes for processing according to the types of each target cluster node and the task types of the tasks to be processed (such as batch processing or real-time computing, etc.), so as to achieve the matching of heterogeneous tasks.
[0064] Correspondingly, the pre-estimated computing power can also be combined with the cloud service provider's pricing strategy to optimize the resource procurement cost. For example, bind long-term and stable database services to reserved instance nodes with stable but low-cost computing power, and use on-demand instance nodes with high but high-cost computing power for burst tasks.
[0065] In this embodiment, by sending a preset task to each target cluster node, the embodiment of the present invention can enable the target cluster node to execute a preset hardware parameter collection script when starting the preset task, and penetrate to collect the underlying hardware parameter information; by calculating the pre-estimated computing power of each target cluster node according to the hardware parameter information, it is possible to comprehensively consider various hardware resources and their synergistic effects, significantly improve the accuracy of computing power estimation of the cluster nodes managed by the cloud management platform, and provide a reliable basis for resource allocation and scheduling of the cloud management platform.
[0066] Corresponding to the above method embodiment, the embodiment of the present invention also provides a device for estimating the computing power of a cluster node. The device for estimating the computing power of a cluster node described below can be correspondingly referred to the method for estimating the computing power of a cluster node described above.
[0067] Please refer to Figure 4 , Figure 4 which is a structural block diagram of a device for estimating the computing power of a cluster node provided by an embodiment of the present invention. The device is applied to a cloud management platform device and may include:
[0068] A task sending module 10, configured to send a preset task to each target cluster node; wherein, the preset task is used to execute a preset hardware parameter collection script when starting;
[0069] A parameter obtaining module 20, configured to obtain the hardware parameter information of each target cluster node; wherein, the hardware parameter information is the information collected by the target cluster node when executing the preset hardware parameter collection script, and the hardware parameter information includes at least two of central processor performance information, memory performance information, and storage performance information;
[0070] The computing power estimation module 30 is configured to calculate the pre-estimated computing power of each target cluster node according to the hardware parameter information.
[0071] In some embodiments, the central processing unit performance information includes at least one of the number of central processing units, the maximum frequency, the base frequency, the number of cores, and the number of threads per core; the memory performance information includes the memory capacity and / or the memory frequency; the storage performance information includes at least one of the storage capacity, the read rate, the write rate, and the storage type.
[0072] In some embodiments, the hardware parameter information includes the central processing unit performance information, the memory performance information, and the storage performance information. The computing power estimation module 30 may include:
[0073] The performance calculation sub-module is configured to calculate the central processing unit performance value C, the memory performance value M, and the storage performance value D of the current cluster node according to the hardware parameter information of the current cluster node; wherein, the current cluster node is any target cluster node; C = num_chips × (w freq1 × (max_freq + base_freq) / 2 + w cores × cores × threads_per_core), M = w volume1 × memory_volume + w freq2 × memory_freq, D = w volume2 × storage_volume + w speed × read_speed + w speed × write_speed; num_chips is the number of central processing units, max_freq is the maximum frequency, base_freq is the base frequency, cores is the number of cores, threads_per_core is the number of threads per core, w freq1 is the first preset frequency weight, w cores is the preset core weight; memory_volume is the memory capacity, memory_freq is the memory frequency, w volume1 is the first preset capacity weight, w freq2 is the second preset frequency weight; storage_volume is the storage capacity, read_speed is the read rate, write_speed is the write rate, w volume2 is the second preset capacity weight, w speed is the preset rate weight;
[0074] The computing power calculation sub-module is configured to calculate the pre-estimated computing power of the current cluster node through w C × C + w M × M + w D × D, where wC is the preset processor weight, w M is the preset memory weight, w D is the preset storage weight, w C + w M + w D = 1, w C > w M > w D .
[0075] In some embodiments, the performance calculation sub-module may include:
[0076] A rate weight determination unit, configured to, if the storage type in the memory performance information of the current cluster node is a solid-state drive, set w speed to adopt a preset solid-state drive weight value; if the storage type in the memory performance information of the current cluster node is not a solid-state drive, then set w speed to adopt a preset other hard drive weight value; wherein, the preset solid-state drive weight value is greater than the preset other hard drive weight value.
[0077] In some embodiments, the preset task includes mount directory information, and the mount directory information includes at least one of a boot program directory, an operating system detailed information directory, a system directory, and a system-level library file storage directory.
[0078] In some embodiments, the parameter acquisition module 20 may be specifically configured to monitor the logs generated during the execution of the preset task by each target cluster node, and acquire the hardware parameter information of each target cluster node.
[0079] In some embodiments, the target cluster node is a node in a container orchestration platform cluster.
[0080] In some embodiments, the computing power estimation module 30 may include:
[0081] A feature label update sub-module, configured to configure the feature label corresponding to each target cluster node according to the hardware parameter information;
[0082] A computing power sub-module, configured to calculate the pre-estimated computing power of each target cluster node according to the value of the feature label.
[0083] In this embodiment, in the embodiment of the present invention, the task distribution module 10 distributes a preset task to each target cluster node, enabling the target cluster node to execute a preset hardware parameter acquisition script when starting the preset task, and penetratingly acquiring the underlying hardware parameter information; by the computing power estimation module 30 calculating the pre-estimated computing power of each target cluster node according to the hardware parameter information, it is possible to comprehensively consider various hardware resources and their synergistic effects, significantly improve the accuracy of computing power estimation of the cluster nodes managed by the cloud management platform, and provide a reliable basis for resource allocation and scheduling of the cloud management platform.
[0084] Corresponding to the above method embodiments, the embodiments of the present invention further provide a cloud management platform device. A cloud management platform device described below can be correspondingly referred to with a computing power estimation method of a cluster node described above.
[0085] Please refer to Figure 5 , Figure 5 which is a schematic diagram of a simple structure of a cloud management platform device provided by an embodiment of the present invention. The cloud management platform device may include:
[0086] A memory D1 for storing a computer program;
[0087] A processor D2 for implementing the steps of the computing power estimation method of the cluster node provided by the above method embodiment when executing the computer program.
[0088] Specifically, please refer to Figure 6 , Figure 6 which is a schematic diagram of a specific structure of a cloud management platform device provided by an embodiment of the present invention. The cloud management platform device 310 may vary greatly due to configuration or performance, and may include one or more processors (central processing units, CPUs) 322 (for example, one or more processors) and a memory 332, and one or more storage media 330 for storing application programs 342 or data 344 (for example, one or more mass storage devices). Among them, the memory 332 and the storage media 330 may be transient storage or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the data processing device. Further, the central processor 322 may be set to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the cloud management platform device 310.
[0089] The cloud management platform device 310 may further include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341. For example, Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
[0090] Among them, the cloud management platform device 310 provided in this embodiment may be a device running a cloud management platform, such as a server.
[0091] The steps in the computing power estimation method of the cluster node described above may be implemented by the structure of the cloud management platform device.
[0092] Corresponding to the above method embodiments, an embodiment of the present invention further provides a computer program product. A computer program product described below can be correspondingly referred to with a method for estimating computing power of a cluster node described above.
[0093] A computer program product includes computer programs / instructions. When the computer programs / instructions are executed by a processor, the steps of the method for estimating computing power of a cluster node provided in the above method embodiments are implemented.
[0094] Corresponding to the above method embodiments, an embodiment of the present invention further provides a computer-readable storage medium. A computer-readable storage medium described below can be correspondingly referred to with a method for estimating computing power of a cluster node described above.
[0095] A computer-readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for estimating computing power of a cluster node in the above method embodiments are implemented.
[0096] Specifically, the computer-readable storage medium can be various readable storage media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.
[0097] The various embodiments in the specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other. For the devices, cloud management platform devices, computer program products, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple. For the relevant parts, reference can be made to the description in the method part.
[0098] The above has introduced in detail a method, a device, and a cloud management platform device for estimating computing power of a cluster node provided by the present invention. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.
Claims
1. A computing power estimation method for cluster nodes, characterized in that Applied to the cloud management platform device, including: Sending a preset task to each target cluster node; wherein, the preset task is used to execute a preset hardware parameter collection script at startup; Obtaining the hardware parameter information of each of the target cluster nodes; wherein, the hardware parameter information is the information collected by the target cluster node when executing the preset hardware parameter collection script, and the hardware parameter information includes at least two of central processor performance information, memory performance information, and storage performance information; Calculating the pre-estimated computing power of each of the target cluster nodes according to the hardware parameter information.
2. The computing power prediction method for the cluster node according to claim 1, wherein The central processor performance information includes at least one of the number of central processors, maximum frequency, base frequency, number of cores, and number of threads per core; The memory performance information includes memory capacity and / or memory frequency; the storage performance information includes at least one of storage capacity, read rate, write rate, and storage type.
3. The computing power prediction method according to claim 2, wherein The hardware parameter information includes the central processor performance information, the memory performance information, and the storage performance information. Calculating the pre-estimated computing power of each of the target cluster nodes according to the hardware parameter information includes: Calculate the central processor performance value C, memory performance value M, and storage performance value D of the current cluster node according to the hardware parameter information of the current cluster node; where the current cluster node is any one of the target cluster nodes; C = num_chips × (w freq1 × (max_freq + base_freq) / 2 + w cores × cores × threads_per_core), M = w volume1 × memory_volume + w freq2 × memory_freq, D = w volume2 × storage_volume + w speed × read_speed + w speed × write_speed; num_chips is the number of central processors, max_freq is the maximum frequency, base_freq is the base frequency, cores is the number of cores, threads_per_core is the number of threads per core, w freq1 is the first preset frequency weight, w cores is the preset core weight; memory_volume is the memory capacity, memory_freq is the memory frequency, w volume1 is the first preset capacity weight, w freq2 is the second preset frequency weight; storage_volume is the storage capacity, read_speed is the read rate, write_speed is the write rate, w volume2 is the second preset capacity weight, w speed is the preset rate weight; Through w C ×C + w M ×M + w D ×D to calculate the pre - estimated force of the current cluster node; where, w C is the preset processor weight, w M is the preset memory weight, w D is the preset storage weight, w C + w M + w D = 1, w C >w M >w D .
4. The computing power estimation method for the cluster node according to claim 3, wherein Calculating the central processor performance value C, memory performance value M, and storage performance value D of the current cluster node according to the hardware parameter information of the current cluster node, including: If the storage type in the memory performance information of the current cluster node is a solid-state drive, then let w speed Adopt a preset solid-state drive weight value; if the storage type in the memory performance information of the current cluster node is not a solid-state drive, then let w speed Adopt a preset weight value for other hard drives; where the preset solid-state drive weight value is greater than the preset weight value for other hard drives.
5. The computing power estimation method of the cluster node according to claim 1, wherein The preset task includes mount directory information, and the mount directory information includes at least one of a boot program directory, an operating system detailed information directory, a system directory, and a system-level library file storage directory.
6. The computing power estimation method for a cluster node according to claim 1, wherein Obtaining the hardware parameter information of each of the target cluster nodes includes: Monitoring the logs generated by each of the target cluster nodes during the execution of the preset task, and obtaining the hardware parameter information of each of the target cluster nodes.
7. The computing power prediction method of the cluster node according to claim 1, wherein The target cluster node is a node in a container orchestration platform cluster.
8. The computing power prediction method for a cluster node according to any one of claims 1 to 7, characterized in that, Calculating the pre-estimated computing power of each of the target cluster nodes according to the hardware parameter information includes: Configuring corresponding feature tags for each of the target cluster nodes according to the hardware parameter information; Calculating the pre-estimated computing power of each of the target cluster nodes according to the values of the feature tags.
9. A computing power estimation device for a cluster node, characterized in that, Applied to the cloud management platform device, including: A task sending module, configured to send a preset task to each target cluster node; wherein, the preset task is used to execute a preset hardware parameter collection script at startup; A parameter acquisition module, configured to obtain the hardware parameter information of each of the target cluster nodes; wherein, the hardware parameter information is the information collected by the target cluster node when executing the preset hardware parameter collection script, and the hardware parameter information includes at least two of central processor performance information, memory performance information, and storage performance information; A computing power estimation module, configured to calculate the pre-estimated computing power of each of the target cluster nodes according to the hardware parameter information.
10. A cloud management platform device, characterized in that, Including: A memory, configured to store a computer program; A processor, configured to implement the steps of the computing power estimation method of the cluster node according to any one of claims 1 to 8 when executing the computer program.