A calculation method, system and terminal for cluster resource planning

Abstract

The present invention discloses a calculation method, system and terminal for cluster resource planning, the method comprising: obtaining the daily data request volume that the cluster needs to carry, using a first preset rule to calculate the query rate per second, and calculating the number of machines according to the query rate per second and the carrying capacity of the cluster; using a second preset rule to calculate the disk size of a single machine according to the daily data request volume, the data volume of each request and the number of machines; using a third preset rule to calculate the memory of each server according to the parameters set by cluster planning; using a fourth preset rule to select the number of CPU cores according to the number of threads; calculating the network card bandwidth according to a fifth preset rule; thereby obtaining a cluster resource planning result. The present invention evaluates cluster resource planning from the dimensions of QPS, memory, disk, CPU, network card bandwidth, etc., thereby obtaining an effective resource evaluation result and avoiding waste of resource application.

Description

Technical Field

[0001] The present invention relates to the field of computer application technology, and in particular to a calculation method, system, terminal and computer-readable storage medium for cluster resource planning. Background Art

[0002] The trunking communication system is a mobile communication system used for group dispatching and commanding communication, and is mainly used in the field of professional mobile communications. The available channels of the system can be shared by all users of the system, and it has the function of automatic channel selection. It is a multi-purpose and high-efficiency wireless dispatching communication system that shares resources, shares costs, and uses channel equipment and services.

[0003] Usually when doing project planning, it is necessary to evaluate server resources, such as how many machines are needed and what the machine configurations are. After evaluation, resource application cost accounting is required, but the existing technology lacks an effective method for evaluating server resources.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide a calculation method, system, terminal and computer-readable storage medium for cluster resource planning, aiming to solve the problem of lack of effective method for evaluating server resources in the prior art.

[0006] To achieve the above object, the present invention provides a calculation method for cluster resource planning, the calculation method for cluster resource planning comprising the following steps:

[0007] Obtain the daily data request volume that the cluster needs to carry, calculate the query rate per second using a first preset rule according to user behavior characteristics, and calculate the number of machines according to the query rate per second and the carrying capacity of the cluster;

[0008] The second preset rule is used to calculate the disk size of a single machine according to the daily data request volume, the data size of each request, and the number of machines;

[0009] Obtain the parameters of the cluster planning settings, and calculate the memory of each server using the third preset rule according to the parameters;

[0010] Get the number of threads in the cluster, and select the number of CPU cores according to the number of threads using the fourth preset rule;

[0011] The fifth preset rule is used to calculate the network card bandwidth based on the daily data request volume, the data size of each request, the number of machines, and the number of replicas;

[0012] The cluster resource planning result is obtained based on the calculated number of machines, disk size of a single machine, memory of each server, number of CPU cores and network card bandwidth.

[0013] The calculation method for cluster resource planning, wherein the daily data request volume that the cluster needs to carry is obtained, the query rate per second is calculated using a first preset rule according to user behavior characteristics, and the number of machines is calculated according to the query rate per second and the carrying capacity of the cluster, specifically including:

[0014] Obtain the daily data request volume a that the cluster needs to carry;

[0015] The first preset rule is: QPS=0.8*a*0.8 / ((24-8)*0.2*60*60);

[0016] Among them, QPS means query rate per second;

[0017] Number of machines m = (QPS / 0.3)*40000.

[0018] The calculation method for cluster resource planning, wherein the second preset rule is used to calculate the disk size of a single machine according to the daily data request volume, the data volume of each request, and the number of machines, specifically includes:

[0019] Get the daily data request volume a, the data size of each request d, and the number of machines m;

[0020] The second preset rule is: single machine disk size = (a*d / (1024*1024*1024))*number of copies*number of days to retain data / m.

[0021] The calculation method for cluster resource planning, wherein the step of obtaining the parameters set for cluster planning and calculating the memory of each server using a third preset rule according to the parameters, specifically includes:

[0022] Get the number of topics, partitions, replicas, log file size, and number of cluster machines set in cluster planning;

[0023] The third preset rule is: memory per server = (number of topics * number of partitions * log file size * number of copies / number of cluster machines) * 0.25.

[0024] The calculation method for cluster resource planning, wherein the step of obtaining the number of threads within the cluster and selecting the number of CPU cores according to the number of threads using a fourth preset rule, specifically includes:

[0025] Estimate the number of threads in the cluster by viewing the source code of the cluster;

[0026] The fourth preset rule is:

[0027] Less than 10 threads: Select 4 CPU cores;

[0028] Between 10-100 threads, select 8 CPU cores;

[0029] Between 100-200 threads, choose 16 CPU cores.

[0030] The calculation method for cluster resource planning, wherein the calculation of the network card bandwidth using the fifth preset rule according to the daily data request volume, the data volume of each request, the number of machines and the number of replicas, specifically includes:

[0031] Get the daily data request volume a, the data size of each request d, the number of machines m, and the number of replicas;

[0032] The fifth preset rule is: network card bandwidth = (QPS / m)*d*number of copies.

[0033] The calculation method for cluster resource planning, wherein the cluster includes a Kafka cluster.

[0034] In addition, to achieve the above-mentioned purpose, the present invention further provides a computing system for cluster resource planning, wherein the computing system for cluster resource planning includes:

[0035] A first calculation module is used to obtain the daily data request volume that the cluster needs to carry, calculate the query rate per second using a first preset rule according to user behavior characteristics, and calculate the number of machines according to the query rate per second and the carrying capacity of the cluster;

[0036] A second calculation module is used to calculate the disk size of a single machine using a second preset rule according to the daily data request volume, the data volume of each request, and the number of machines;

[0037] A third calculation module is used to obtain parameters set in cluster planning, and calculate the memory of each server using a third preset rule according to the parameters;

[0038] A fourth calculation module, used to obtain the number of threads in the cluster, and select the number of CPU cores according to the number of threads using a fourth preset rule;

[0039] A fifth calculation module, used to calculate the network card bandwidth using a fifth preset rule according to the daily data request volume, the data volume of each request, the number of machines and the number of replicas;

[0040] The result generation module is used to obtain the cluster resource planning result according to the calculated number of machines, the disk size of a single machine, the memory of each server, the number of CPU cores and the network card bandwidth.

[0041] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a cluster resource planning calculation program stored in the memory and executable on the processor, and when the cluster resource planning calculation program is executed by the processor, the steps of the cluster resource planning calculation method as described above are implemented.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a calculation program for cluster resource planning, and when the calculation program for cluster resource planning is executed by a processor, the steps of the calculation method for cluster resource planning as described above are implemented.

[0043] The present invention obtains the daily data request amount that the cluster needs to carry, calculates the query rate per second using the first preset rule according to the user behavior characteristics, and calculates the number of machines according to the query rate per second and the carrying capacity of the cluster; uses the second preset rule to calculate the disk size of a single machine according to the daily data request amount, the data size of each request and the number of machines; obtains the parameters set by the cluster planning, and uses the third preset rule to calculate the memory of each server according to the parameters; obtains the number of threads inside the cluster, and uses the fourth preset rule to select the number of CPU cores according to the number of threads; uses the fifth preset rule to calculate the network card bandwidth according to the daily data request amount, the data size of each request, the number of machines and the number of copies; obtains the cluster resource planning result according to the calculated number of machines, the disk size of a single machine, the memory of each server, the number of CPU cores and the network card bandwidth. The present invention evaluates the cluster resource planning from the dimensions of QPS, memory, disk, CPU, network card bandwidth, etc., so as to obtain effective resource evaluation results and avoid waste of resource application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of a preferred embodiment of the calculation method of cluster resource planning of the present invention;

[0045] Figure 2 It is a schematic diagram of the principle of a preferred embodiment of a computing system for cluster resource planning of the present invention;

[0046] Figure 3 Schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] The calculation method of cluster resource planning described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the calculation method of cluster resource planning includes the following steps:

[0049] Step S10: Obtain the daily data request volume that the cluster needs to carry, calculate the query rate per second using the first preset rule according to the user behavior characteristics, and calculate the number of machines according to the query rate per second and the carrying capacity of the cluster.

[0050] Specifically, obtain the daily data request volume a that the cluster (for example, the present invention takes the Kafka cluster as an example. Kafka is an open source stream processing platform developed by the Apache Software Foundation and is written in Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that can process all action stream data of consumers on the website. In addition to the Kafka cluster, other clusters can also flexibly use the method of the present invention) needs to carry; the first preset rule is: QPS=0.8*a*0.8 / ((24-8)*0.2*60*60); wherein QPS represents the query rate per second; the number of machines m=(QPS / 0.3)*40000.

[0051] For example, suppose the cluster needs to carry 1 billion data requests every day, 24 hours a day. For the user behavior characteristics of the OTT (Over The Top, Internet companies bypass operators to develop various video and data service businesses based on the open Internet. This is a very popular term in the communications industry. This term comes from sports such as basketball and means "overhead pass", which means that basketball players pass the ball back and forth above their heads to reach the destination. That is, Internet companies bypass operators to develop various video and data service businesses based on the open Internet, emphasizing the independence of services from physical networks) industry, there is almost no user data in the 8 hours from 12 o'clock in the evening to 8 o'clock in the morning. Then, using the 80 / 20 rule, it is estimated that 80% of the data (800 million) will flow in during the remaining 16 hours, and 80% of the 800 million data (640 million) will flow in during 20% ​​of the 16 hours (3 hours).

[0052] The calculation formula (i.e., the first preset rule) for QPS (query rate per second, which is a measure of the amount of traffic processed by a specific query server within a specified time) is as follows:

[0053] Assuming the daily data request volume is a, then:

[0054] QPS=0.8*a*0.8 / ((24-8)*0.2*60*60)

[0055] Here a=1000000000, 640000000 / (3*60*60)=60,000, so during peak period, the Kafka cluster needs to withstand 60,000 concurrent requests per second.

[0056] In order to control the peak QPS within 30% of the total QPS that the cluster can carry, the total QPS that the Kafka cluster can currently carry is 200,000 to be safe. Since one physical machine can support 40,000 QPS, from the perspective of QPS, 5-7 physical machines are required, and the number of machines = (QPS / 0.3)*40000.

[0057] Step S20: Calculate the disk size of a single machine using a second preset rule according to the daily data request volume, the data volume of each request, and the number of machines.

[0058] Specifically, obtain the daily data request amount a, the data amount of each request d and the number of machines m; the second preset rule is: single machine disk size = (a*d / (1024*1024*1024))*number of copies*data retention days / m.

[0059] For example, assuming that the daily data request volume is a, the data volume of each request is d kb, and the number of machines is m, then the single machine disk size calculation formula (i.e., the second preset rule) = (a*d / (1024*1024*1024))*number of copies*number of days to retain data / m.

[0060] For example, if there are 1 billion data items per day and each request is 50kb, that is 46TB of data. If you save a copy, 46*2=92TB, and keep the data from the last 3 days. So 92*3=276TB is needed. 5 physical machines need to store 276TB of data, and each machine stores 60TB of data. The general configuration is 11 disks, and each disk requires 7TB.

[0061] Step S30: Obtain parameters set by cluster planning, and calculate the memory of each server according to the parameters using a third preset rule.

[0062] Specifically, the number of topics, number of partitions, number of copies, log file size and number of cluster machines set in the cluster planning are obtained; the third preset rule is: memory per server = (number of topics * number of partitions * log file size * number of copies / number of cluster machines) * 0.25.

[0063] For example, the memory of each server is calculated according to the number of topics, partitions, replicas, log file size, and number of cluster machines (nodes) set in the Kafka cluster planning. The calculation formula (i.e., the third preset rule) is as follows:

[0064] Memory per server = (number of topics * number of partitions * log file size * number of replicas / number of machines in the cluster) * 0.25.

[0065] For example, the company cluster sets up 100 topics, and the partition data of these 100 topics is best stored in the OS cache. There are 100 topics, and each topic has 5 partitions, so there will be a total of 500 partitions. The log file size of each partition is 1G, and there are 2 copies, which means that it takes 1000G of memory to store all the partition data of 100 topics in the memory. There are now 5 servers, so on average the server needs 200G of memory every day, but in fact, it is not necessary for all the partition data to reside in the memory. Only 25% of the data is needed in the memory, 200G*0.25=50G. So a total of 56G of memory is needed, so you can choose a server with 64G memory.

[0066] Step S40: Obtain the number of threads in the cluster, and select the number of CPU cores according to the number of threads using a fourth preset rule.

[0067] Specifically, the number of threads in the cluster is estimated by viewing the source code of the cluster; the fourth preset rule is:

[0068] Less than 10 threads: Select 4 CPU cores;

[0069] Between 10-100 threads, select 8 CPU cores;

[0070] Between 100-200 threads, choose 16 CPU cores.

[0071] In other words, CPU planning mainly depends on how many threads there are in the Kafka process. Threads are mainly executed by multi-core CPUs. If there are too many threads but too few CPU cores, the CPU load will be very high, which will lead to low efficiency of overall work thread execution. By looking at the Kafka source code, it is estimated that there are more than 100 threads in Kafka. According to the above rules, 16 cores are selected.

[0072] Step S50: Calculate the network card bandwidth using the fifth preset rule according to the daily data request volume, the data volume of each request, the number of machines and the number of replicas.

[0073] Specifically, the daily data request amount a, the data amount of each request d, the number of machines m and the number of replicas are obtained; the fifth preset rule is: network card bandwidth = (QPS / m)*d*number of replicas.

[0074] For example, during peak hours, there will be about 60,000 requests per second, and each machine will process about 10,000 requests. Each request is 50kb, so about 488M of data will come in per second, and there is also replica synchronization data. Therefore, during peak hours, 488M*2=976M / s of network bandwidth is required. Therefore, during peak hours, at least Gigabit bandwidth must be used.

[0075] Step S60: Obtain a cluster resource planning result based on the calculated number of machines, disk size of a single machine, memory of each server, number of CPU cores, and network card bandwidth.

[0076] Specifically, the cluster resource planning results include the calculated number of machines, the disk size of a single machine, the memory of each server, the number of CPU cores and the network card bandwidth, for example, 1 billion requests, 6w / s throughput, 276T of data, 5 physical machines; the hard disk is 11 (SAS)*7T, 7200 rpm; the memory is 64GB; the CPU is 16 cores; the network is a gigabit network card.

[0077] This article uses the Kafka cluster as an example, mainly including QPS estimation, calculating the number of machines required for the cluster, the number of disks required for a single machine, server memory selection, CPU selection based on the thread number range standard, and network card bandwidth selection based on QPS. It is a general cluster resource estimation method that avoids waste of resource application and controls project budget.

[0078] Furthermore, if Figure 2 As shown, based on the above-mentioned calculation method for cluster resource planning, the present invention also provides a calculation system for cluster resource planning, wherein the calculation system for cluster resource planning includes:

[0079] A first calculation module 51 is used to obtain the daily data request volume that the cluster needs to carry, calculate the query rate per second using a first preset rule according to user behavior characteristics, and calculate the number of machines according to the query rate per second and the carrying capacity of the cluster;

[0080] A second calculation module 52, configured to calculate the disk size of a single machine using a second preset rule according to the daily data request volume, the data volume of each request, and the number of machines;

[0081] A third calculation module 53 is used to obtain parameters set in cluster planning, and calculate the memory of each server according to the parameters using a third preset rule;

[0082] A fourth calculation module 54 is used to obtain the number of threads in the cluster and select the number of CPU cores according to the number of threads using a fourth preset rule;

[0083] A fifth calculation module 55, configured to calculate the network card bandwidth using a fifth preset rule according to the daily data request volume, the data volume of each request, the number of machines and the number of replicas;

[0084] The result generation module 56 is used to obtain the cluster resource planning result according to the calculated number of machines, the disk size of a single machine, the memory of each server, the number of CPU cores and the network card bandwidth.

[0085] Furthermore, if Figure 3 As shown, based on the above-mentioned calculation method and system for cluster resource planning, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 3 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0086] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a calculation program 40 for cluster resource planning is stored on the memory 20, and the calculation program 40 for cluster resource planning can be executed by the processor 10, thereby realizing the calculation method for cluster resource planning in the present application.

[0087] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the calculation method of the cluster resource planning.

[0088] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0089] In one embodiment, when the processor 10 executes the computing program 40 for cluster resource planning in the memory 20, the following steps are implemented:

[0090] Obtain the daily data request volume that the cluster needs to carry, calculate the query rate per second using a first preset rule according to user behavior characteristics, and calculate the number of machines according to the query rate per second and the carrying capacity of the cluster;

[0091] The second preset rule is used to calculate the disk size of a single machine according to the daily data request volume, the data size of each request, and the number of machines;

[0092] Obtain the parameters of the cluster planning settings, and calculate the memory of each server using the third preset rule according to the parameters;

[0093] Get the number of threads in the cluster, and select the number of CPU cores according to the number of threads using the fourth preset rule;

[0094] The fifth preset rule is used to calculate the network card bandwidth based on the daily data request volume, the data size of each request, the number of machines, and the number of replicas;

[0095] The cluster resource planning result is obtained based on the calculated number of machines, disk size of a single machine, memory of each server, number of CPU cores and network card bandwidth.

[0096] The obtaining of the daily data request volume that the cluster needs to carry, calculating the query rate per second using a first preset rule according to the user behavior characteristics, and calculating the number of machines according to the query rate per second and the carrying capacity of the cluster specifically include:

[0097] Obtain the daily data request volume a that the cluster needs to carry;

[0098] The first preset rule is: QPS=0.8*a*0.8 / ((24-8)*0.2*60*60);

[0099] Among them, QPS means query rate per second;

[0100] Number of machines m = (QPS / 0.3)*40000.

[0101] The method of calculating the disk size of a single machine using a second preset rule according to the daily data request volume, the data volume of each request, and the number of machines specifically includes:

[0102] Get the daily data request volume a, the data size of each request d, and the number of machines m;

[0103] The second preset rule is: single machine disk size = (a*d / (1024*1024*1024))*number of copies*number of days to retain data / m.

[0104] The step of obtaining the parameters set in cluster planning and calculating the memory of each server using a third preset rule according to the parameters specifically includes:

[0105] Get the number of topics, partitions, replicas, log file size, and number of cluster machines set in cluster planning;

[0106] The third preset rule is: memory per server = (number of topics * number of partitions * log file size * number of copies / number of cluster machines) * 0.25.

[0107] The obtaining of the number of threads within the cluster and selecting the number of CPU cores according to the number of threads using a fourth preset rule specifically includes:

[0108] Estimate the number of threads in the cluster by viewing the source code of the cluster;

[0109] The fourth preset rule is:

[0110] Less than 10 threads: Select 4 CPU cores;

[0111] Between 10-100 threads, select 8 CPU cores;

[0112] Between 100-200 threads, choose 16 CPU cores.

[0113] The method of calculating the network card bandwidth using the fifth preset rule according to the daily data request volume, the data volume of each request, the number of machines and the number of replicas specifically includes:

[0114] Get the daily data request volume a, the data size of each request d, the number of machines m, and the number of replicas;

[0115] The fifth preset rule is: network card bandwidth = (QPS / m)*d*number of copies.

[0116] Wherein, the cluster includes a Kafka cluster.

[0117] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a calculation program for cluster resource planning, and when the calculation program for cluster resource planning is executed by a processor, the steps of the calculation method for cluster resource planning as described above are implemented.

[0118] In summary, the present invention provides a calculation method, system and terminal for cluster resource planning, the method comprising: obtaining the daily data request volume that the cluster needs to carry, calculating the query rate per second using the first preset rule according to the user behavior characteristics, and calculating the number of machines according to the query rate per second and the carrying capacity of the cluster; calculating the disk size of a single machine using the second preset rule according to the daily data request volume, the data volume of each request and the number of machines; obtaining the parameters set by the cluster planning, and calculating the memory of each server using the third preset rule according to the parameters; obtaining the number of threads inside the cluster, and selecting the number of CPU cores according to the number of threads using the fourth preset rule; calculating the network card bandwidth using the fifth preset rule according to the daily data request volume, the data volume of each request, the number of machines and the number of copies; obtaining the cluster resource planning result according to the calculated number of machines, the disk size of a single machine, the memory of each server, the number of CPU cores and the network card bandwidth. The present invention evaluates cluster resource planning from the dimensions of QPS, memory, disk, CPU, network card bandwidth, etc., so as to obtain effective resource evaluation results and avoid waste of resource application.

[0119] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0120] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0121] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A calculation method for cluster resource planning, characterized in that: The calculation method of cluster resource planning includes: Obtain the daily data request volume that the cluster needs to carry, calculate the query rate per second using a first preset rule according to user behavior characteristics, and calculate the number of machines according to the query rate per second and the carrying capacity of the cluster; The second preset rule is used to calculate the disk size of a single machine according to the daily data request volume, the data size of each request, and the number of machines; Obtain the parameters of the cluster planning settings, and calculate the memory of each server using the third preset rule according to the parameters; Get the number of threads in the cluster, and select the number of CPU cores according to the number of threads using the fourth preset rule; The fifth preset rule is used to calculate the network card bandwidth based on the daily data request volume, the data size of each request, the number of machines, and the number of replicas; The cluster resource planning result is obtained based on the calculated number of machines, disk size of a single machine, memory of each server, number of CPU cores and network card bandwidth.

2. The calculation method for cluster resource planning according to claim 1, characterized in that: The obtaining of the daily data request volume that the cluster needs to carry, calculating the query rate per second using a first preset rule according to the user behavior characteristics, and calculating the number of machines according to the query rate per second and the carrying capacity of the cluster, specifically includes: Obtain the daily data request volume a that the cluster needs to carry; The first preset rule is: QPS=0.8*a*0.8 / ((24-8)*0.2*60*60); Among them, QPS means query rate per second; Number of machines m = (QPS / 0.3)*40000.

3. The calculation method for cluster resource planning according to claim 2, characterized in that: The method of calculating the disk size of a single machine using a second preset rule according to the daily data request volume, the data volume of each request, and the number of machines includes: Get the daily data request volume a, the data size of each request d, and the number of machines m; The second preset rule is: single machine disk size = (a*d / (1024*1024*1024))*number of copies*number of days to retain data / m.

4. The calculation method for cluster resource planning according to claim 3, characterized in that: The step of obtaining the parameters set in cluster planning and calculating the memory of each server using a third preset rule according to the parameters specifically includes: Get the number of topics, partitions, replicas, log file size, and number of cluster machines set in cluster planning; The third preset rule is: memory per server = (number of topics * number of partitions * log file size * number of copies / number of cluster machines) * 0.

25.

5. The calculation method for cluster resource planning according to claim 1, characterized in that: The obtaining of the number of threads within the cluster and selecting the number of CPU cores according to the number of threads using a fourth preset rule specifically includes: Estimate the number of threads in the cluster by viewing the source code of the cluster; The fourth preset rule is: Less than 10 threads: Select 4 CPU cores; Between 10-100 threads, select 8 CPU cores; Between 100-200 threads, choose 16 CPU cores.

6. The calculation method for cluster resource planning according to claim 4, characterized in that: The fifth preset rule is used to calculate the network card bandwidth according to the daily data request volume, the data volume of each request, the number of machines and the number of replicas, specifically including: Get the daily data request volume a, the data size of each request d, the number of machines m, and the number of replicas; The fifth preset rule is: network card bandwidth=(QPS / m)*d*number of copies.

7. The calculation method for cluster resource planning according to any one of claims 1 to 6, characterized in that: The cluster includes a Kafka cluster.

8. A computing system for cluster resource planning, characterized in that: The computing system for cluster resource planning includes: A first calculation module is used to obtain the daily data request volume that the cluster needs to carry, calculate the query rate per second using a first preset rule according to user behavior characteristics, and calculate the number of machines according to the query rate per second and the carrying capacity of the cluster; A second calculation module is used to calculate the disk size of a single machine using a second preset rule according to the daily data request volume, the data volume of each request, and the number of machines; A third calculation module is used to obtain parameters set in cluster planning, and calculate the memory of each server according to the parameters using a third preset rule; A fourth calculation module, used to obtain the number of threads in the cluster, and select the number of CPU cores according to the number of threads using a fourth preset rule; A fifth calculation module, used to calculate the network card bandwidth using a fifth preset rule according to the daily data request volume, the data volume of each request, the number of machines and the number of replicas; The result generation module is used to obtain the cluster resource planning result according to the calculated number of machines, the disk size of a single machine, the memory of each server, the number of CPU cores and the network card bandwidth.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a cluster resource planning calculation program stored in the memory and executable on the processor. When the cluster resource planning calculation program is executed by the processor, the steps of the cluster resource planning calculation method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a calculation program for cluster resource planning, and when the calculation program for cluster resource planning is executed by a processor, the steps of the calculation method for cluster resource planning according to any one of claims 1 to 7 are implemented.

Images