Massive cluster optimization method, device, apparatus and storage medium

Abstract

The application provides a large-scale cluster optimization method and device, equipment and a storage medium, which are applied to a control node. The control node is used for running an application programming interface service component of a large-scale cluster. The control node acquires a node quantity change state corresponding to the large-scale cluster and a load state of the application programming interface service component. Whether to adjust a node reporting frequency is determined according to the node quantity change state and the load state. If yes, a target node reporting frequency parameter is acquired, and the node reporting frequency is adjusted according to the target node reporting frequency parameter. The application intelligently adjusts the node reporting frequency by monitoring and analyzing the cluster node quantity change and the load state of the API service component in real time, thereby effectively reducing the load pressure of the control node and improving the overall performance and stability of the cluster.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large-scale cluster, and in particular, to a large-scale cluster optimization method, device, equipment and storage medium. BACKGROUND

[0002] With the wide application of cloud native technology, Kubernetes (K8S) has become the preferred platform for managing containerized applications. In order to cope with the growing business needs, enterprises or organizations continue to expand the scale of K8S cluster and increase the number of Node nodes to provide more computing resources.

[0003] In the related art, when facing a large-scale K8S cluster (especially when the number of Node nodes exceeds 1000), the pressure on the control plane increases, which may cause the control plane to collapse, thereby affecting the scheduling and business operation of the cluster, and failing to ensure the high availability and elastic scaling capability of the cluster, thereby failing to guarantee the continuity and stability of the business. SUMMARY

[0004] Therefore, the present application aims to provide a large-scale cluster optimization method, device, equipment and storage medium to solve the problem that the high availability and elastic scaling capability of the cluster cannot be ensured due to the expansion of the cluster, thereby failing to guarantee the continuity and stability of the business.

[0005] According to a first aspect of the present application, a large-scale cluster optimization system is provided, applied to a control node, wherein the control node is used to run an application programming interface service component of a large-scale cluster, and the method comprises:

[0006] obtaining a node quantity change state corresponding to the large-scale cluster and a load state of the application programming interface service component;

[0007] determining whether to adjust the node reporting frequency according to the node quantity change state and the load state;

[0008] if yes, obtaining a target node reporting frequency parameter and adjusting the node reporting frequency according to the target node reporting frequency parameter.

[0009] Optionally, before the step of obtaining the target node reporting frequency parameter, the method further comprises:

[0010] determining whether the large-scale cluster enables a node lease feature.

[0011] Optionally, the step of obtaining the target node reporting frequency parameter comprises:

[0012] in the case of detecting that the node lease feature corresponding to the large-scale cluster is in an unenabled state, obtaining a configuration parameter;

[0013] determining the target node reporting frequency parameter according to the configuration parameter.

[0014] Optionally, the configuration parameter is an integer multiple of the target node reporting frequency parameter.

[0015] Optionally, the obtaining the target node reporting frequency parameter comprises:

[0016] in a case where it is detected that the node lease feature corresponding to the large-scale cluster is in an enabled state, obtaining the target node reporting frequency parameter according to a current node quantity corresponding to the large-scale cluster.

[0017] Optionally, the adjusting the node reporting frequency according to the target node reporting frequency parameter comprises:

[0018] adjusting the node reporting frequency according to a frequency value of the target node reporting frequency parameter.

[0019] Optionally, the obtaining the node quantity change state and the load state of the application programming interface service component comprises:

[0020] querying, by a first preset command line tool, a first node quantity corresponding to the large-scale cluster at a starting time point in a preset time period, and a second node quantity corresponding to the large-scale cluster at a terminal time point in the preset time period;

[0021] determining the node quantity change state according to the first node quantity and the second node quantity, wherein the node quantity change state comprises an increase state and a decrease state;

[0022] querying, by a second preset command line tool, resource usage data of the application programming interface service component of the large-scale cluster in the preset time period, and determining a load state according to the resource usage data;

[0023] wherein the determining the node quantity change state according to the first node quantity and the second node quantity comprises:

[0024] if the second node quantity is greater than the first node quantity, the node quantity change state is the increase state;

[0025] if the second node quantity is less than the first node quantity, the node quantity change state is the decrease state;

[0026] wherein the determining the load state according to the resource usage data comprises:

[0027] determining a load state according to a change of the resource usage data in the preset time period, the load state comprising a load increase state;

[0028] The determining whether to adjust the node reporting frequency according to the node quantity change state and the load state comprises:

[0029] In a case where it is detected that the node quantity change state is the increasing state and the load state is the load increasing state, it is determined to adjust the node reporting frequency.

[0030] According to a second aspect of the present application, there is provided a large-scale cluster optimization apparatus applied to a control node, the control node being configured to run an application programming interface service component of a large-scale cluster, the apparatus comprising:

[0031] an obtaining module configured to obtain a node quantity change state corresponding to the large-scale cluster and a load state of the application programming interface service component;

[0032] a determining module configured to determine whether to adjust a node reporting frequency according to the node quantity change state and the load state;

[0033] an adjusting module configured to, if so, obtain a target node reporting frequency parameter and adjust the node reporting frequency according to the target node reporting frequency parameter.

[0034] Optionally, the apparatus further comprises:

[0035] a judging module configured to judge whether the large-scale cluster enables a node lease feature.

[0036] Optionally, the adjusting module comprises:

[0037] a first adjusting submodule configured to, in a case where it is detected that a node lease feature corresponding to the large-scale cluster is in a disabled state, obtain a configuration parameter;

[0038] a second adjusting submodule configured to determine the target node reporting frequency parameter according to the configuration parameter.

[0039] Optionally, the second adjusting submodule is specifically configured to make the configuration parameter an integer multiple of the target node reporting frequency parameter.

[0040] Optionally, the adjusting module comprises:

[0041] a third adjusting submodule configured to, in a case where it is detected that a node lease feature corresponding to the large-scale cluster is in an enabled state, obtain a target node reporting frequency parameter according to a current node quantity corresponding to the large-scale cluster.

[0042] Optionally, the adjusting module comprises:

[0043] The fourth adjusting sub-module is configured to adjust the node reporting frequency according to a frequency value of a frequency parameter reported by the target node.

[0044] Optionally, the obtaining module comprises:

[0045] The first obtaining sub-module is configured to query, by using a first preset command line tool, a first node quantity corresponding to the large-scale cluster at a starting time point in a preset time period and a second node quantity corresponding to the large-scale cluster at a terminal time point in the preset time period.

[0046] The second obtaining sub-module is configured to determine the node quantity change state according to the first node quantity and the second node quantity, wherein the node quantity change state comprises an increasing state and a decreasing state.

[0047] The third obtaining sub-module is configured to query, by using a second preset command line tool, resource usage data of the application programming interface service component of the large-scale cluster in the preset time period, and determine a load state according to the resource usage data.

[0048] The second obtaining sub-module comprises:

[0049] The first obtaining unit is configured to determine that the node quantity change state is the increasing state if the second node quantity is greater than the first node quantity.

[0050] The second obtaining unit is configured to determine that the node quantity change state is the decreasing state if the second node quantity is less than the first node quantity.

[0051] The third obtaining sub-module comprises:

[0052] The load unit is configured to determine a load state according to a change of the resource usage data in the preset time period, wherein the load state comprises a load increasing state.

[0053] The determining module comprises:

[0054] The determining sub-module is configured to determine to adjust the node reporting frequency if it is detected that the node quantity change state is the increasing state and the load state is the load increasing state.

[0055] According to another aspect of the present application, an electronic device is also provided, comprising:

[0056] a processor;

[0057] a memory for storing instructions executable by the processor;

[0058] The processor is configured to execute the instructions to implement the large-scale cluster optimization method as described above.

[0059] According to a further aspect of the present application, there is also provided a readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for optimizing a large-scale cluster as described above.

[0060] The method for optimizing a large-scale cluster provided by the embodiments of the present application is applied to a control node, and the control node is used to run an application programming interface service component of the large-scale cluster. The control node acquires a node quantity change state corresponding to the large-scale cluster and a load state of the application programming interface service component. The control node determines whether to adjust a node reporting frequency according to the node quantity change state and the load state. If yes, the control node acquires a target node reporting frequency parameter and adjusts the node reporting frequency according to the target node reporting frequency parameter. The present application adjusts the node reporting frequency intelligently by monitoring and analyzing the node quantity change of the cluster and the load state of the API service component in real time, thereby effectively reducing the load pressure of the control node and improving the overall performance and stability of the cluster. The optimization strategy can ensure that the control node can still run efficiently in the case of continuous expansion of the cluster size, avoid performance bottlenecks or system crashes caused by excessive load, guarantee the continuity and high availability of the business, and improve the resource utilization efficiency and response speed, thereby providing strong support for the stable operation of the large-scale K8S cluster.

[0061] The above description is only a summary of the technical solutions of the present application. In order to enable one skilled in the art to better understand the technical means of the present application, the content of the specification can be implemented, and in order to enable the above and other purposes, features and advantages of the present application to be more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0062] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not meant to limit the present application. Furthermore, the same reference numerals are used throughout the several views to denote the same or similar parts. In the drawings:

[0063] Figure 1 is one of the step flowcharts of the method for optimizing a large-scale cluster provided by the embodiments of the present application;

[0064] Figure 2 is another of the step flowcharts of the method for optimizing a large-scale cluster provided by the embodiments of the present application;

[0065] Figure 3 is a third of the step flowcharts of the method for optimizing a large-scale cluster provided by the embodiments of the present application;

[0066] Figure 4Figure 4 is a fourth step flow chart of a large-scale cluster optimization method provided by an embodiment of the present application;

[0067] Figure 5 Figure 1 is a device block diagram of a large-scale cluster optimization device provided by an embodiment of the present application;

[0068] Figure 6 Figure 5 is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0069] Figure 7 Figure 6 is a cluster architecture schematic diagram of a large-scale cluster optimization method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0070] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are presented in order to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and based on various changes and modifications of the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the embodiments can be combined and referenced with each other without contradiction.

[0071] Referring to Figure 1 , Figure 1 shows a first step flow chart of a large-scale cluster optimization method provided by an embodiment of the present application, the method can include:

[0072] Step 101, obtaining a node quantity change state corresponding to the large-scale cluster and a load state of the application programming interface service component;

[0073] It should be noted that, in the embodiments of the present application, the large-scale cluster can be a K8S cluster with more than 1000 nodes, the K8S cluster is divided into control nodes and computing nodes, the control nodes run components or services of the K8S control plane, such as apiserver, controller-manager, scheduler, etcd, kubelet, etc., the computing nodes run components or services such as kubelet, proxy, calico, etc., and the basic architecture of the large-scale cluster K8S cluster is as shown in Figure 7

[0074] ​The computing node is managed by the control plane component, the computing node state needs to be reported to the control plane component in real time, and the services running thereon also need to be scheduled and managed by the control plane. Therefore, the increasing scale of the cluster node will bring pressure to the K8S native control component. Among them, the K8S core API component k8s-apiserver is the first to be perceived. K8s apiserver is the gateway of the Kubernetes cluster. It is accessible to all users, automation and components in the Kubernetes cluster. APIServer implements RESTfuI API through HTTP, performs all API operations, and is responsible for storing API objects to the persistent storage backend.

[0075] When the K8S cluster increases by one node, the necessary components kubelet, proxy, calico, etc. on the node will be added accordingly. The node establishes an interface call connection with the apiserver through these components and continuously reports the state to the apiserver in real time. After receiving the reporting message, the apiserver will call the persistent interface to update the node information to the persistent layer.

[0076] With the continuous increase of the number of nodes, the frequency of calling apiserver is increasing, so the pressure of this component will continue to rise. Since the cluster nodes report periodically, the pressure on apiserver does not decrease, which affects the calling of apiserver by other processes. At the same time, apiserver does not set a limit on resources, so the increasing resource consumption will affect the load of the control node of the K8S cluster, which may affect the normal operation of the control node component and further affect the entire cluster.

[0077] Therefore, the load of the application programming interface service component can be associated with the scale of the number of nodes, that is, when the scale of the number of nodes increases, the load of apiserver increases. When each node needs to report state information and service information running on the node to apiserver in real time, the calling of apiserver by nodes is uninterrupted for a large-scale cluster (more than 1000 nodes), so reducing the frequency of calling apiserver by nodes will have a decompression effect.

[0078] Therefore, in the embodiments of the present application, first, the node number change state corresponding to the large-scale cluster and the load state of the application programming interface service component are obtained. By analyzing the number of nodes of the current cluster and the load of apiserver, it is determined whether the node reporting frequency needs to be adjusted.

[0079] Further, with reference to Figure 4 , Figure 4The step 101 includes steps 1011-1013, i.e.

[0080] In step 1011, a first preset command line tool is used to query a first node number corresponding to the large-scale cluster at a starting time point in a preset time period, and a second node number corresponding to the large-scale cluster at an ending time point in the preset time period.

[0081] In step 1012, the node number change state is determined according to the first node number and the second node number, where the node number change state includes an increase state and a decrease state.

[0082] Further, in step 1012, if the second node number is greater than the first node number, the node number change state is the increase state.

[0083] If the second node number is less than the first node number, the node number change state is the decrease state.

[0084] In step 1013, a second preset command line tool is used to query resource usage data of the application programming interface service component of the large-scale cluster in the preset time period, and a load state is determined according to the resource usage data.

[0085] Further, in step 1013, the determination of the load state according to the resource usage data includes: determining the load state according to a change of the resource usage data in the preset time period, and the load state includes a load increase state.

[0086] It should be noted that, in the embodiments of the present application, steps 1011 and 1012 monitor and analyze the node number change of the large-scale cluster, use a first preset command line tool to query a first node number at a starting time point and a second node number at an ending time point in a preset time period, and then determine the node number change state, i.e. the increase state or the decrease state, according to the two data points. This method helps to grasp the change trend of the cluster size in real time, and provides data support for subsequent resource management and optimization decisions.

[0087] Specifically, the kubectl command line tool is used to query the node number in the cluster. The following command is executed: kubectl get nodes, which lists all nodes in the cluster and displays the number of nodes.

[0088] Therefore, the node number change state can be determined to be the increase state or the decrease state according to the node number change in a period of time.

[0089] In step 1013, the resource usage of the API service component in the large-scale cluster is monitored by a second preset command-line tool, and the load state is evaluated based on these data. Specifically, this step involves collecting resource usage data of the API service component within a preset time period, such as CPU, memory, network, etc., and then analyzing the trend of these data to determine whether the load state is in the elevated state. This method can help administrators understand the performance status of the API service component in real time, discover and respond to possible load pressure in a timely manner, and ensure the stable operation and high-performance service of the cluster.

[0090] Specifically, the kubectl top command can be used, and if the Metrics Server is enabled in the cluster, the following command can be used to view the resource usage of the apiserver: kubectl top pods --namespace=kube-system | grep kube-apiserver, which will display the CPU and memory usage of the apiserver pod.

[0091] If Prometheus and Grafana monitoring systems are deployed in the large-scale cluster, the load state of the apiserver can be viewed through the Grafana dashboard. In Grafana, there are predefined dashboards that display the request rate, response time, error rate, and other indicators of the apiserver. The apiserver logs can be viewed by executing the following command: kubectl logs -n kube-system <apiserver-pod-name>wherein, <apiserver-pod-name>is the name of the apiserver pod.

[0092] You can also use system monitoring tools (such as top, htop, nmon, etc.) on the cluster nodes to view the resource usage of the apiserver process. Log in to the node running the apiserver and execute the following command: top -p <apiserver-pid>wherein, <apiserver-pid>is the PID of the apiserver process.

[0093] In step 102, it is determined whether to adjust the node reporting frequency according to the node quantity change state and the load state.

[0094] Further, the determination of whether to adjust the node reporting frequency according to the node quantity change state and the load state comprises:

[0095] In the case where the node quantity change state is detected as the increasing state and the load state is detected as the load increasing state, it is determined to adjust the node reporting frequency.

[0096] Therefore, in step 102, in the case where the node quantity change state is detected as the increasing state and the load state is detected as the load increasing state, it is determined to adjust the node reporting frequency.

[0097] In step 103, if yes, a target node reporting frequency parameter is obtained, and the node reporting frequency is adjusted according to the target node reporting frequency parameter.

[0098] Further, step 103, i.e., the adjustment of the node reporting frequency according to the target node reporting frequency parameter comprises:

[0099] The node reporting frequency is adjusted according to the frequency value of the target node reporting frequency parameter.

[0100] It should be noted that in the embodiments of the present application, the target node reporting frequency parameter is nodeStatusUpdateFrequency, and the apiserver load increasing problem caused by the increase of the cluster node number needs to reduce the reporting frequency of the cluster node to the apiserver, and therefore, the cluster node reporting frequency parameter nodeStatusUpdateFrequency needs to be modified.

[0101] Further, the state reporting of the cluster node is completed by kubelet calling the apiserver of the k8s cluster, and therefore, the kubelet parameters of all nodes of the cluster need to be uniformly adjusted, the reporting frequency is prolonged, and the frequency of the cluster node reporting to the apiserver is reduced.

[0102] Wherein, the kubelet parameter is a target node reporting frequency parameter, specifically, modifying the kubelet configuration file is logging into each cluster node, finding the kubelet configuration file, wherein the configuration file can be located in the / etc / kubernetes / manifest path or other customized path, adding or modifying the nodeStatusUpdateFrequency parameter in the kubelet configuration file, and setting it to a proper value (format: string).

[0103] The large-scale cluster optimization method provided by the embodiment of the application is applied to a control node, and the control node is used to run an application programming interface service component of a large-scale cluster. The node quantity change state corresponding to the large-scale cluster and the load state of the application programming interface service component are obtained. Whether the node reporting frequency is adjusted is determined according to the node quantity change state and the load state. If yes, a target node reporting frequency parameter is obtained, and the node reporting frequency is adjusted according to the target node reporting frequency parameter. The node reporting frequency is intelligently adjusted by real-time monitoring and analyzing the cluster node quantity change and the load state of the API service component, so that the load pressure of the control node is effectively reduced, and the overall performance and stability of the cluster are improved. This optimization strategy can ensure that the control node can still be efficiently run in the case that the cluster scale is continuously expanded, avoid performance bottlenecks or system crashes caused by excessive load, guarantee the continuity and high availability of the business, improve the resource utilization efficiency and response speed, and provide strong support for the stable operation of the large-scale K8S cluster.

[0104] In addition, in the embodiment of the application, after the node reporting frequency is adjusted according to the target node reporting frequency parameter, the kubelet service can be restarted on each control node to make the changed parameters take effect, and the load state of the apiserver can be further monitored to verify whether the node reporting frequency has been reduced, and ensure that the cluster runs normally. The detailed information of the configuration change is recorded, and the related documents are updated for future reference and maintenance.

[0105] Referring to Figure 2 , a second step flowchart of the large-scale cluster optimization method provided by the embodiment of the application is shown, and the method can include:

[0106] In step 201, the node quantity change state corresponding to the large-scale cluster and the load state of the application programming interface service component are obtained.

[0107] In step 202, whether the node reporting frequency is adjusted is determined according to the node quantity change state and the load state.

[0108] Step 203, if yes, it is judged whether the large-scale cluster enables the node lease feature;

[0109] Step 204, in the case of detecting that the node lease feature corresponding to the large-scale cluster is in the disabled state, a configuration parameter is acquired;

[0110] Step 205, the target node reporting frequency parameter is determined according to the configuration parameter, and the node reporting frequency is adjusted according to the target node reporting frequency parameter.

[0111] It should be noted that in the embodiment of the present application, before adjustment, it is also necessary to determine whether the current large-scale cluster enables the node lease feature.

[0112] If the node lease feature is not enabled, the target node reporting frequency parameter setting is also the frequency of kubelet delivering node status to the control plane, therefore, if the node lease feature is not enabled, the set value must be coordinated with the configuration parameter of the node controller, wherein the configuration parameter is nodeMonitorGracePeriod.

[0113] Further, the configuration parameter is an integer multiple of the target node reporting frequency parameter.

[0114] It should be noted that on the basis of adjusting the nodeStatusUpdateFrequency parameter, the configuration parameter nodeMonitorGracePeriod of the k8s controller manager component also needs to be modified, and in the embodiment of the present application, it is necessary to ensure that nodeMonitorGracePeriod is an integer multiple of nodeStatudUpdateFrequency.

[0115] Specifically, nodeMonitorGracePeriod is a configuration parameter of the node controller (NodeController) in Kubernetes, which defines the time that the node controller waits before considering a node unreachable. Specifically, this parameter specifies the length of time that the node controller waits for kubelet to report the node status before marking the node status as NotReady or Unknown.

[0116] In the Kubernetes cluster, the node controller is responsible for monitoring the status of all nodes in the cluster. Kubelet periodically reports the status of the node it is in to the API Server, and if the node controller does not receive a status update from a certain node within nodeMonitorGracePeriod, it considers the node unreachable and marks its status as NotReady or Unknown.

[0117] For example, if nodeMonitorGracePeriod is set to "40s", the node controller will mark a node as unreachable if it does not receive a status update from that node within 40 seconds.

[0118] It is important to note that the value of nodeMonitorGracePeriod should be set in coordination with the kubelet's nodeStatusUpdateFrequency parameter. Generally, nodeMonitorGracePeriod should be an integer multiple of nodeStatusUpdateFrequency to ensure that the node controller has enough time to process node status updates. For example, if nodeStatusUpdateFrequency is set to "10s", nodeMonitorGracePeriod can be set to "40s" (i.e., 4 times nodeStatusUpdateFrequency).

[0119] By reasonably setting these two parameters, we can ensure that the Kubernetes cluster can respond in a timely manner when the node status changes, while avoiding false labeling of node status due to temporary network problems or other temporary failures.

[0120] The present application intelligently adjusts the node reporting frequency by real-time monitoring and analyzing the cluster node quantity change and the load state of API service components, thereby effectively reducing the load pressure of the control node and improving the overall performance and stability of the cluster. This optimization strategy can ensure that the control node can still run efficiently as the cluster size continues to expand, avoiding performance bottlenecks or system crashes due to excessive load, ensuring business continuity and high availability, while also improving resource utilization efficiency and response speed, providing strong support for the stable operation of large-scale K8S clusters.

[0121] Referring to Figure 3 , a third step flow chart of a large-scale cluster optimization method provided by an embodiment of the present application is shown, which can include:

[0122] Step 301, obtaining the node quantity change state corresponding to the large-scale cluster and the load state of the application programming interface service component;

[0123] Step 302, determining whether to adjust the node reporting frequency according to the node quantity change state and the load state;

[0124] Step 303, if yes, determining whether the large-scale cluster enables node lease characteristics;

[0125] In step 304, when it is detected that the node lease feature corresponding to the large-scale cluster is in an enabled state, a target node reporting frequency parameter is obtained according to a current node quantity corresponding to the large-scale cluster, and the node reporting frequency is adjusted according to the target node reporting frequency parameter.

[0126] In the case where it is determined that the node lease feature corresponding to the large-scale cluster is in an enabled state, the value of the node reporting frequency parameter nodeStatusUpdateFrequency can be adjusted according to the node quantity scale, and the format is string. If the parameter does not exist, it means that the original configuration of the K8S cluster is used without any modification. At this time, the node reporting frequency can be actively adjusted by adding the parameter.

[0127] The application intelligently adjusts the node reporting frequency by monitoring and analyzing the cluster node quantity change and the load state of the API service component in real time, thereby effectively reducing the load pressure of the control node and improving the overall performance and stability of the cluster. This optimization strategy can ensure that the control node can still run efficiently in the case of continuous expansion of the cluster scale, avoid performance bottlenecks or system crashes caused by excessive load, and guarantee the continuity and high availability of the business, while improving the resource utilization efficiency and response speed, providing strong support for the stable operation of the large-scale K8S cluster.

[0128] The embodiment of the application also provides a large-scale cluster optimization device, which refers to Figure 5 , Figure 5 is a device block diagram of a large-scale cluster optimization device provided by the embodiment of the application, and the large-scale cluster optimization device comprises:

[0129] The acquisition module 501 is configured to acquire a node quantity change state corresponding to the large-scale cluster and a load state of an application programming interface service component.

[0130] The determination module 502 is configured to determine whether to adjust a node reporting frequency according to the node quantity change state and the load state.

[0131] The adjustment module 503 is configured to, if yes, acquire a target node reporting frequency parameter and adjust the node reporting frequency according to the target node reporting frequency parameter.

[0132] Optionally, the device further comprises:

[0133] The judgment module is configured to judge whether the large-scale cluster enables a node lease feature.

[0134] Optionally, the adjustment module comprises:

[0135] The first adjusting sub-module is configured to acquire a configuration parameter when it is detected that the node lease feature corresponding to the large-scale cluster is in the disabled state.

[0136] The second adjusting sub-module is configured to determine the target node reporting frequency parameter according to the configuration parameter.

[0137] Optionally, the second adjusting sub-module is specifically configured to make the configuration parameter an integer multiple of the target node reporting frequency parameter.

[0138] Optionally, the adjusting module comprises:

[0139] The third adjusting sub-module is configured to acquire a target node reporting frequency parameter according to a current node quantity corresponding to the large-scale cluster when it is detected that the node lease feature corresponding to the large-scale cluster is in the enabled state.

[0140] Optionally, the adjusting module comprises:

[0141] The fourth adjusting sub-module is configured to adjust the node reporting frequency according to a frequency value of the target node reporting frequency parameter.

[0142] Optionally, the acquiring module comprises:

[0143] The first acquiring sub-module is configured to query a first node quantity corresponding to the large-scale cluster at a starting time point in a preset time period and a second node quantity corresponding to the large-scale cluster at a terminal time point in the preset time period through a first preset command line tool;

[0144] The second acquiring sub-module is configured to determine the node quantity change state according to the first node quantity and the second node quantity, wherein the node quantity change state comprises an increase state and a decrease state.

[0145] The third acquiring sub-module is configured to query resource usage data of the application programming interface service component of the large-scale cluster in the preset time period through a second preset command line tool and determine a load state according to the resource usage data.

[0146] Optionally, the second acquiring sub-module comprises:

[0147] The first acquiring unit is configured to determine that the node quantity change state is the increase state if the second node quantity is greater than the first node quantity.

[0148] The second acquiring unit is configured to determine that the node quantity change state is the decrease state if the second node quantity is less than the first node quantity.

[0149] Optionally, the third acquiring sub-module comprises:

[0150] a load unit configured to determine a load state according to a change of the resource usage data within the preset time period, the load state including a load increase state;

[0151] The determination module includes:

[0152] A determination sub-module is configured to, in a case where it is detected that the node quantity change state is the increase state and the load state is the load increase state, determine to adjust a node reporting frequency.

[0153] The application can effectively reduce the load pressure of the control node and improve the overall performance and stability of the cluster by monitoring and analyzing the cluster node quantity change and the load state of the API service component in real time and intelligently adjusting the node reporting frequency. This optimization strategy can ensure that the control node can still operate efficiently even as the cluster size continues to expand, avoiding performance bottlenecks or system crashes caused by excessive load, ensuring the continuity and high availability of the business, and improving resource utilization efficiency and response speed, thereby providing strong support for the stable operation of large-scale K8S clusters.

[0154] The application also provides an electronic device, such as Figure 6 As shown in the figure, the electronic device includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 complete mutual communication through the communication bus 604,

[0155] The memory 603 is used to store a computer program.

[0156] The processor 601 is used to execute the program stored in the memory 603, and the following steps are implemented:

[0157] obtain a node quantity change state corresponding to the large-scale cluster and a load state of an application programming interface service component;

[0158] determine whether to adjust a node reporting frequency according to the node quantity change state and the load state;

[0159] If yes, obtain a target node reporting frequency parameter, and adjust the node reporting frequency according to the target node reporting frequency parameter.

[0160] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0161] The communication interface is used for communication between the terminal and other devices.

[0162] The memory can include a Random Access Memory (RAM) and can also include a non-volatile memory, such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the processor.

[0163] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0164] In another embodiment provided by the application, a computer readable storage medium is also provided, and the computer readable storage medium stores instructions, when the instructions are run on a computer, the computer executes the large-scale cluster optimization in any of the above embodiments.

[0165] In the embodiments described above, all or some of the steps can be implemented by hardware, software, firmware or any combination thereof. When implemented in software, all or some of the steps can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, all or some of the steps generate the processes or functions described in the embodiments of the present application. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0166] It should be noted that, in this document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a…" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0167] Each of the embodiments in the specification is described in a related manner, and the same or similar parts between each of the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0168] The above merely provides the preferred embodiments of the application, and not intended to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall fall within the protection scope of the application.

Claims

1. A method for large scale cluster optimization, the method comprising: The method is applied to a control node used for running an application programming interface service component of a large-scale cluster, and comprises the following steps: obtaining a node quantity change state corresponding to the large-scale cluster and a load state of the application programming interface service component; determining whether to adjust a node reporting frequency according to the node quantity change state and the load state; if yes, obtaining a target node reporting frequency parameter and adjusting the node reporting frequency according to the target node reporting frequency parameter; wherein, before the step of obtaining the target node reporting frequency parameter, the method further comprises: judging whether a node lease feature corresponding to the large-scale cluster is enabled; the step of obtaining the target node reporting frequency parameter comprises: in the case that it is detected that the node lease feature corresponding to the large-scale cluster is in a disabled state, obtaining a configuration parameter; determining the target node reporting frequency parameter according to the configuration parameter; in the case that it is detected that the node lease feature corresponding to the large-scale cluster is in an enabled state, obtaining a target node reporting frequency parameter according to a current node quantity corresponding to the large-scale cluster; the step of obtaining the node quantity change state corresponding to the large-scale cluster and the load state of the application programming interface service component comprises: inquiring, through a first preset command line tool, a first node quantity corresponding to the large-scale cluster at a starting time point in a preset time period and a second node quantity corresponding to the large-scale cluster at a terminal time point in the preset time period; determining the node quantity change state according to the first node quantity and the second node quantity, wherein the node quantity change state comprises an increasing state and a decreasing state; inquiring, through a second preset command line tool, resource usage data of the application programming interface service component of the large-scale cluster in the preset time period, and determining a load state according to the resource usage data; wherein, the step of determining the node quantity change state according to the first node quantity and the second node quantity comprises: if the second node quantity is greater than the first node quantity, the node quantity change state is the increasing state; if the second node quantity is less than the first node quantity, the node quantity change state is the decreasing state; wherein, the step of determining the load state according to the resource usage data comprises: determining a load state according to a change of the resource usage data in the preset time period, wherein the load state comprises a load increasing state; the step of determining whether to adjust the node reporting frequency according to the node quantity change state and the load state comprises: in the case that it is detected that the node quantity change state is the increasing state and the load state is the load increasing state, it is determined to adjust the node reporting frequency.

2. The method of claim 1, wherein, the configuration parameter is an integer multiple of the target node reporting frequency parameter.

3. The method of claim 1, wherein, the step of adjusting the node reporting frequency according to the target node reporting frequency parameter comprises: adjusting the node reporting frequency according to a frequency value of the target node reporting frequency parameter.

4. A large-scale cluster optimization apparatus characterized by comprising: The device is applied to a control node used for running an application programming interface service component of a large-scale cluster, and comprises: The acquisition module is configured to acquire a node quantity change state corresponding to the large-scale cluster and a load state of the application programming interface service component; The determination module is configured to determine whether to adjust a node reporting frequency according to the node quantity change state and the load state; The adjustment module is configured to, if yes, acquire a target node reporting frequency parameter and adjust the node reporting frequency according to the target node reporting frequency parameter; The judgment module is configured to judge whether the large-scale cluster enables a node lease feature; The adjustment module comprises: The first adjustment submodule is configured to acquire a configuration parameter in a case where it is detected that the node lease feature corresponding to the large-scale cluster is in an unenabled state; The second adjustment submodule is configured to determine the target node reporting frequency parameter according to the configuration parameter; The third adjustment submodule is configured to acquire a target node reporting frequency parameter according to a current node quantity corresponding to the large-scale cluster in a case where it is detected that the node lease feature corresponding to the large-scale cluster is in an enabled state; The acquisition module comprises: The first acquisition submodule is configured to query a first node quantity corresponding to the large-scale cluster at a starting time point in a preset time period and a second node quantity corresponding to the large-scale cluster at a terminal time point in the preset time period through a first preset command line tool; The second acquisition submodule is configured to determine the node quantity change state according to the first node quantity and the second node quantity, wherein the node quantity change state comprises an increase state and a decrease state; The third acquisition submodule is configured to query resource usage data of the application programming interface service component of the large-scale cluster in the preset time period through a second preset command line tool and determine a load state according to the resource usage data; The second acquisition submodule comprises: The first acquisition unit is configured to, if the second node quantity is greater than the first node quantity, determine that the node quantity change state is the increase state; The second acquisition unit is configured to, if the second node quantity is less than the first node quantity, determine that the node quantity change state is the decrease state; The third acquisition submodule comprises: The load unit is configured to determine a load state according to a change of the resource usage data in the preset time period, and the load state comprises a load increase state; The determination module comprises: The determination submodule is configured to, in a case where it is detected that the node quantity change state is the increase state and the load state is the load increase state, determine to adjust the node reporting frequency.

5. An electronic device, comprising: comprise: a memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the large-scale cluster optimization method according to any one of claims 1 to 3.

6. A readable storage medium characterized by, The computer program stored on the readable storage medium is executed by the processor to implement the large-scale cluster optimization method according to any one of claims 1 to 3.

Images