Method and apparatus for optimizing watch request in ultra-large-scale kubernetes cluster

Abstract

The present application relates to the technical field of cloud native. Disclosed are a method and apparatus for optimizing a Watch request in an ultra-large-scale Kubernetes cluster. The method comprises: generating a resource version number view in a Gateway component, and broadcasting the resource version number view to all Operators, wherein the resource version number view comprises a correspondence between the Operators and resource version numbers (step S11); and when the Operators execute Watch operations, on the basis of the resource version number view, searching for an Operator that holds a resource version number corresponding to a Watch resource object, and sending a Watch request to the searched Operator (step S12).

Description

A method and apparatus for optimizing Watch requests in ultra-large-scale Kubernetes clusters

[0001] Related applications

[0002] This application claims priority to Chinese Patent Application No. 202411639450.5, filed on November 15, 2024, entitled "A Method and Apparatus for Optimizing Watch Requests in a Large-Scale Kubernetes Cluster", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of cloud-native technology, specifically to a method and apparatus for optimizing Watch requests in ultra-large-scale Kubernetes clusters. Background Technology

[0004] In the practical application of building ultra-large-scale Kubernetes clusters, the Kubelet, controller, and operator on each node frequently access the Kubernetes API server via Watch requests to monitor resource changes, such as creating new Pods, deleting Pods, or changing Pod states. Each Watch request consumes the API server's computing resources and network bandwidth. When the cluster size reaches tens of thousands and the number of resource objects reaches millions, a large number of concurrent Watch requests can cause the API server to crash due to insufficient resources or overload, leading to cluster unavailability or stability issues. Summary of the Invention

[0005] This application provides a method and apparatus for optimizing Watch requests in ultra-large-scale Kubernetes clusters.

[0006] In a first aspect, this application provides a method for optimizing Watch requests in ultra-large-scale Kubernetes clusters, executed by a computer device, the method comprising:

[0007] In the Gateway component, a resource version number view is generated and broadcast to all Operators. The resource version number view includes the correspondence between Operators and resource version numbers.

[0008] When an Operator performs a Watch operation, it looks up the Operator containing the resource version number corresponding to the Watch resource object based on the resource version number view, and sends a Watch request to the found Operator.

[0009] In one alternative implementation, generating a resource version number view in the Gateway component includes:

[0010] The Gateway component is used to manage the resource objects of all Operator Watches in the cluster; and,

[0011] A resource version number view is generated based on the resource object of the Operator Watch and the resource version number corresponding to the resource object.

[0012] In an optional implementation, the method further includes:

[0013] When an Operator fails to perform a Watch operation, it searches for an Operator with the same resource version number as the Watch resource object based on the resource version number view, and sends a Watch request to the found Operator.

[0014] In an optional implementation, the method further includes:

[0015] Update the resource version number view in the Gateway component periodically;

[0016] And, broadcast the updated resource version number view to all Operators.

[0017] In an optional implementation, the method further includes:

[0018] Control the Operator to periodically report the resource version number of the current resource object to the Gateway component.

[0019] In an alternative implementation, the Gateway component adapts to the gateway protocol of the Kubernetes cluster and proxies cluster traffic.

[0020] Secondly, this application provides a device for optimizing watch requests in ultra-large-scale Kubernetes clusters, the device comprising:

[0021] The broadcast module is used to generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators. The resource version number view includes the correspondence between Operators and resource version numbers; and,

[0022] The query module is used to find the Operator with the resource version number corresponding to the Watch resource object when the Operator performs the Watch operation, and send a Watch request to the found Operator.

[0023] In one optional implementation, the broadcast module includes:

[0024] The management unit is used to manage the resource objects of all Operator Watches in the cluster using the Gateway component; and,

[0025] The generation unit is used to generate a resource version number view based on the resource object of the Operator Watch and the resource version number corresponding to the resource object.

[0026] Thirdly, this application provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the ultra-large-scale Kubernetes cluster watch request optimization method described in the first aspect or any corresponding embodiment.

[0027] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute the ultra-large-scale Kubernetes cluster Watch request optimization method described in the first aspect or any of its corresponding embodiments. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the disclosed drawings without creative effort.

[0029] Figure 1 is a schematic diagram of the existing Kubernetes cluster architecture;

[0030] Figure 2 is a flowchart illustrating the method for optimizing Watch requests in a large-scale Kubernetes cluster according to an embodiment of this application;

[0031] Figure 3 is a schematic diagram of a Kubernetes cluster architecture according to an embodiment of this application;

[0032] Figure 4 is a schematic diagram of another Kubernetes cluster architecture according to an embodiment of this application;

[0033] Figure 5 is a structural block diagram of a large-scale Kubernetes cluster watch request optimization device according to an embodiment of this application;

[0034] Figure 6 is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application. Detailed Implementation

[0035] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] The definitions are as follows:

[0037] Kubernetes is an open-source container orchestration engine used to automate the deployment, scaling, and management of containerized applications. Kubelet is one of the core components of Kubernetes, responsible for managing container execution on a single node. Operator is Kubernetes extension software that leverages custom resources to manage applications and their components. APIserver is the gateway to the Kubernetes cluster; it's the central contact point accessible to all users, automations, and components within the Kubernetes cluster. Controller is used to control Pods. The watch mechanism is a real-time resource state change notification mechanism based on HTTP long polling. A Pod represents a process running in the cluster. Etcd is a distributed key-value store system in the Kubernetes cluster used to store all cluster data. Scheduler is the scheduler. Gateway is an instance of traffic processing infrastructure (such as a cloud load balancer).

[0038] In the practice of building ultra-large-scale Kubernetes clusters, the Kubelet, Controller, and Operator on each node frequently access the Kubernetes API server via Watch requests to monitor resource changes, such as creating new Pods, deleting Pods, or changing Pod states. Each Watch request consumes the API server's computing resources and network bandwidth. When the cluster size reaches tens of thousands and the number of resource objects reaches millions, a large number of concurrent Watch requests can cause the API server to crash due to insufficient resources or overload, leading to cluster unavailability or stability issues. The current situation is shown in Figure 1.

[0039] To address existing problems in the construction of ultra-large-scale Kubernetes clusters, this application provides a method for optimizing Watch requests in ultra-large-scale Kubernetes clusters. By reducing the number of Watch requests to the API server, the pressure on the API server is alleviated, ensuring the stability and performance of the cluster.

[0040] According to an embodiment of this application, an embodiment of a method for optimizing Watch requests in a large-scale Kubernetes cluster is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0041] This embodiment provides a method for optimizing Watch requests in a large-scale Kubernetes cluster, which can be used on the aforementioned mobile terminals, such as mobile phones and tablets. Figure 2 is a flowchart of the method for optimizing Watch requests in a large-scale Kubernetes cluster according to an embodiment of this application. As shown in Figure 2, the process includes the following steps:

[0042] Step S11: Generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators. The resource version number view includes the correspondence between Operators and resource version numbers.

[0043] Specifically, as shown in Figure 3, the self-developed Gateway component manages all resource objects watched by operators in the cluster, generates corresponding resource version number (ResourceVersion) views, and broadcasts these views to all operators watching the same resource object. This way, each operator listening to the same resource can obtain the ResourceVersion views of other operators. The Gateway component adapts to the Kubernetes cluster's gateway protocol and proxies cluster traffic.

[0044] Step S12: When the Operator performs the Watch operation, the Operator with the resource version number corresponding to the Watch resource object is found in the resource version number view, and a Watch request is sent to the found Operator.

[0045] Specifically, as shown in Figure 3, when an optimized Operator needs to perform a Watch operation, it will find the Operator with the ResourceVersion corresponding to the resource object it needs to watch based on the ResourceVersion view of the resource object obtained from the Gateway, and send a Watch request to this Operator instead of sending a request to the API server. This significantly reduces the number of Watch requests to the API server in the cluster, alleviates the burden on the API server, and enhances the stability and robustness of large-scale Kubernetes clusters.

[0046] This application provides a method for optimizing Watch requests in ultra-large-scale Kubernetes clusters, including: generating a resource version number view in the Gateway component and broadcasting the resource version number view to all Operators; the resource version number view includes the correspondence between Operators and resource version numbers; when an Operator performs a Watch operation, it looks up the Operator with the resource version number corresponding to the Watch resource object based on the resource version number view, and sends a Watch request to the found Operator. A self-developed Gateway component is introduced to uniformly manage cluster traffic, ensuring a more even distribution of Kubernetes cluster traffic. Furthermore, it coordinates running Operators to ensure that Operators watching the same resource have the same ResourceVersion view. The optimized Operator receives the ResourceVersion view from the self-developed Gateway. When a Watch operation is needed, it selects another Operator based on the view to send a Watch request, reducing the number of Watch requests to the API server, alleviating the pressure on the API server, and ensuring cluster stability and performance.

[0047] In one optional implementation, step S11 includes the following steps:

[0048] Step S111: Use the Gateway component to manage the resource objects of all Operator Watches in the cluster.

[0049] Step S112: Generate a resource version number view based on the resource object of Operator Watch and the resource version number corresponding to the resource object.

[0050] Specifically, the Gateway component is responsible for managing all resource objects that operators in the cluster need to watch, and generating a resource version number view based on the resource objects watched by the operators and the resource version numbers corresponding to the resource objects.

[0051] In one alternative implementation, the method further includes the following steps:

[0052] Step S13: When the Operator fails to execute the Watch operation, the Operator with the same resource version number as the Watch resource object is found in the resource version number view, and a Watch request is sent to the found Operator.

[0053] Specifically, as shown in Figure 4, when the current Watch operation fails due to network failures or other reasons and a new Watch request needs to be made, the system selects the same Operator as the ResourceVersion of the resource that needs to be watched, based on its own saved ResourceVersion view, and sends the Watch request to this Operator.

[0054] In one alternative implementation, the method further includes the following steps:

[0055] Step S14: Update the resource version number view in the Gateway component periodically.

[0056] Step S15: Broadcast the updated resource version number view to all Operators.

[0057] Step S16: Control the Operator to periodically report the resource version number of the current resource object to the Gateway component.

[0058] Specifically, the Gateway component periodically updates the ResourceVersion view of resource objects and broadcasts it to all Operators watching the current resource, so that each Operator listening to the same resource can obtain the ResourceVersion views of other Operators. Operators periodically report the ResourceVersion of the current resource object to the self-developed Gateway component, and receive and save the ResourceVersion views of resource objects sent from the self-developed Gateway component.

[0059] In one alternative implementation, the embodiments of this application can also be applied to the following practical scenarios:

[0060] (1) This application enables a single cluster node to reach 10,000 and run stably, solving the problem of difficult management caused by customer service being scattered in multiple independent Kubernetes clusters, reducing cluster management and maintenance costs, and improving the overall integrity and availability of the cluster.

[0061] (2) In the construction of large-scale Kubernetes clusters, the introduction of this application can effectively reduce the pressure on the Apiserver service, further improve service performance, and ensure that stable service response can still be provided under high load, thereby meeting the ever-growing business needs of customers.

[0062] This embodiment also provides a device for optimizing watch requests in a large-scale Kubernetes cluster. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0063] This embodiment provides a device for optimizing watch requests in a large-scale Kubernetes cluster, as shown in Figure 5, including:

[0064] Broadcast module 51 is used to generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators. The resource version number view includes the correspondence between Operators and resource version numbers.

[0065] The query module 52 is used to find the Operator with the resource version number corresponding to the Watch resource object based on the resource version number view when the Operator performs the Watch operation, and send a Watch request to the found Operator.

[0066] In one alternative implementation, the broadcast module 51 includes:

[0067] Management unit 511 is used to manage the resource objects of all Operator Watches in the cluster using the Gateway component;

[0068] The generation unit 512 is used to generate a resource version number view based on the resource object of Operator Watch and the resource version number corresponding to the resource object.

[0069] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0070] In this embodiment, the ultra-large-scale Kubernetes cluster Watch request optimization device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0071] This application provides a large-scale Kubernetes cluster Watch request optimization device. It introduces a self-developed Gateway component to uniformly manage cluster traffic, ensuring a more even distribution of Kubernetes cluster traffic. Furthermore, it coordinates running Operators, ensuring that Operators watching the same resource have the same ResourceVersion view. The optimized Operator receives the ResourceVersion view from the self-developed Gateway. When a Watch operation is needed, it selects another Operator based on the view to send a Watch request, reducing the number of Watch requests to the API server, alleviating the pressure on the API server, and ensuring cluster stability and performance.

[0072] This application embodiment also provides a computer device having the ultra-large-scale Kubernetes cluster Watch request optimization device shown in FIG5 above.

[0073] Please refer to Figure 6, which is a schematic diagram of the structure of a computer device provided in an optional embodiment of this application. As shown in Figure 6, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be installed on a common motherboard or otherwise as needed. The processor can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 uses one processor 10 as an example.

[0074] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0075] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0076] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0077] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0078] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means; Figure 6 shows an example of a connection via a bus.

[0079] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0080] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0081] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for optimizing Watch requests in ultra-large-scale Kubernetes clusters, wherein, Performed by a computer device, the method includes: In the Gateway component, a resource version number view is generated and broadcast to all Operators. This resource version number view includes the correspondence between Operators and resource version numbers; and... When an Operator performs a Watch operation, it looks up the Operator containing the resource version number corresponding to the Watch resource object based on the resource version number view, and sends a Watch request to the found Operator.

2. The method for optimizing Watch requests in ultra-large-scale Kubernetes clusters according to claim 1, wherein, Generate a resource version number view in the Gateway component, including: The Gateway component is used to manage the resource objects of all Operator Watches in the cluster; and, A resource version number view is generated based on the resource object of the Operator Watch and the resource version number corresponding to the resource object.

3. The method for optimizing Watch requests in ultra-large-scale Kubernetes clusters according to claim 1, wherein, The method further includes: When an Operator fails to perform a Watch operation, it searches for an Operator with the same resource version number as the Watch resource object based on the resource version number view, and sends a Watch request to the found Operator.

4. The method for optimizing Watch requests in ultra-large-scale Kubernetes clusters according to claim 1, wherein, The method further includes: The resource version number view in the Gateway component is updated periodically; and, Broadcast the updated resource version number view to all Operators.

5. The method for optimizing Watch requests in ultra-large-scale Kubernetes clusters according to claim 1, wherein, The method further includes: Control the Operator to periodically report the resource version number of the current resource object to the Gateway component.

6. The method for optimizing Watch requests in ultra-large-scale Kubernetes clusters according to claim 1, wherein, The Gateway component adapts to the gateway protocol of the Kubernetes cluster and proxies cluster traffic.

7. A device for optimizing watch requests in ultra-large-scale Kubernetes clusters, wherein, The device includes: The broadcast module is used to generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators. The resource version number view includes the correspondence between Operators and resource version numbers; and, The query module is used to find the Operator with the resource version number corresponding to the Watch resource object when the Operator performs the Watch operation, and send a Watch request to the found Operator.

8. The ultra-large-scale Kubernetes cluster Watch request optimization device according to claim 7, wherein, The broadcast module includes: The management unit is used to manage the resource objects of all Operator Watches in the cluster using the Gateway component; and, The generation unit is used to generate a resource version number view based on the resource object of the Operator Watch and the resource version number corresponding to the resource object.

9. A computer device, wherein, include: A memory and a processor are interconnected, the memory stores computer instructions, and the processor executes the computer instructions to perform the ultra-large-scale Kubernetes cluster watch request optimization method according to any one of claims 1 to 6.

10. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer instructions for causing the computer to execute the ultra-large-scale Kubernetes cluster Watch request optimization method as described in any one of claims 1 to 6.

Images