Super-large scale Kubernetes cluster Watch request optimization method and device

Abstract

The invention relates to the technical field of cloud native, and discloses a super-large scale Kubernetes cluster Watch request optimization method and device, and the method comprises the steps: generating a resource version number view in a Gateway assembly, and broadcasting the resource version number view to all Operators, the resource version number view comprising a corresponding relation between the Operators and resource version numbers; and when the Operator executes the Watch operation, searching the Operator where the resource version number corresponding to the Watch resource object is located according to the resource version number view, and sending a Watch request to the searched Operator. By reducing the number of Watch requests for accessing the Apiserver, the pressure of the Apiserver is relieved, and the stability and performance of the cluster are ensured.

Description

Technical Field

[0001] The present invention relates to the field of cloud native technology, and in particular to a method and device for optimizing Watch requests of an ultra-large-scale Kubernetes cluster. Background Art

[0002] In the practice of building ultra-large-scale Kubernetes clusters, the Kubelet, control plane Controller, and Operator of each node in the cluster frequently access the Kubernetes Apiserver through Watch requests to monitor resource changes, such as creating new Pods, deleting Pods, or changing Pod status. Each Watch request will occupy the computing resources and network bandwidth of the Apiserver. When the cluster node scale reaches tens of thousands and the resource objects reach millions, a large number of concurrent Watch requests will cause the Apiserver to crash due to insufficient resources or overload, causing cluster unavailability or stability issues. Summary of the invention

[0003] In view of this, the present invention provides a method and device for optimizing Watch requests in a super-large-scale Kubernetes cluster to solve the problem of cluster instability or unavailability caused by excessive load on the Kubernetes Apiserver due to a large number of Watch requests in the cluster.

[0004] In a first aspect, the present invention provides a method for optimizing Watch requests in a super-large-scale Kubernetes cluster, the method comprising:

[0005] Generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators, where the resource version number view includes the correspondence between Operators and resource version numbers;

[0006] When the Operator performs a Watch operation, it searches for the Operator where the resource version number corresponding to the Watch resource object is located according to the resource version number view, and sends a Watch request to the found Operator.

[0007] The present invention provides a method for optimizing Watch requests for ultra-large-scale Kubernetes clusters. The method introduces a self-developed Gateway component, which is responsible for unified management of cluster traffic to ensure that the Kubernetes cluster traffic is more evenly distributed. In addition, the running Operators are coordinated to ensure that the Operators that watch the same resource have the same ResourceVersion view. The optimized Operator receives the ResourceVersion view issued by the self-developed Gateway. When a Watch operation is required, other Operators are selected according to the view to send Watch requests, thereby reducing the number of Watch requests to access the Apiserver, alleviating the pressure on the Apiserver and ensuring the stability and performance of the cluster.

[0008] In an optional implementation, generating a resource version number view in a Gateway component includes:

[0009] Use the Gateway component to manage the resource objects of all Operator Watches in the cluster;

[0010] Generate a resource version number view based on the resource object of the Operator Watch and the resource version number corresponding to the resource object.

[0011] In an optional embodiment, the method further includes:

[0012] When the Operator fails to perform the Watch operation, it searches for an Operator with the same resource version number as the Watch resource object according to the resource version number view, and sends a Watch request to the found Operator.

[0013] In an optional embodiment, the method further includes:

[0014] Regularly update the resource version number view in the Gateway component;

[0015] Broadcast the updated resource version number view to all Operators.

[0016] In an optional embodiment, the method further includes:

[0017] Controls the Operator to periodically report the resource version number of the current resource object to the Gateway component.

[0018] In an optional implementation, the Gateway component adapts the gateway protocol of the Kubernetes cluster and proxies the cluster traffic.

[0019] In a second aspect, the present invention provides a device for optimizing Watch requests in a super-large-scale Kubernetes cluster, the device comprising:

[0020] A broadcast module, used to generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators, wherein the resource version number view includes a correspondence between Operators and resource version numbers;

[0021] The query module is used to search the Operator where the resource version number corresponding to the Watch resource object is located according to the resource version number view when the Operator performs the Watch operation, and send a Watch request to the found Operator.

[0022] In an optional implementation, the broadcast module includes:

[0023] Management unit, used to manage resource objects of all Operator Watches in the cluster using the Gateway component;

[0024] The generation unit is used to generate a resource version number view according to the resource object of the Operator Watch and the resource version number corresponding to the resource object.

[0025] The present invention provides a super-large-scale Kubernetes cluster Watch request optimization device, which introduces a self-developed Gateway component and is responsible for unified management of cluster traffic to ensure that the Kubernetes cluster traffic is more evenly distributed. In addition, it coordinates the running Operators to ensure that the Operators that Watch the same resource have the same ResourceVersion view. The optimized Operator receives the ResourceVersion view issued by the self-developed Gateway. When a Watch operation is required, it selects other Operators to send Watch requests based on the view, reducing the number of Watch requests to access the Apiserver, alleviating the pressure on the Apiserver, and ensuring the stability and performance of the cluster.

[0026] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the ultra-large-scale Kubernetes cluster Watch request optimization method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0027] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the ultra-large-scale Kubernetes cluster Watch request optimization method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 This is a schematic diagram of the existing Kubernetes cluster architecture;

[0030] Figure 2 It is a flowchart of a method for optimizing Watch requests for a super-large-scale Kubernetes cluster according to an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of a Kubernetes cluster architecture according to an embodiment of the present invention;

[0032] Figure 4 is another schematic diagram of a Kubernetes cluster architecture according to an embodiment of the present invention;

[0033] Figure 5 is a structural block diagram of a super-large-scale Kubernetes cluster Watch request optimization device according to an embodiment of the present invention;

[0034] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0036] The glossary is explained as follows:

[0037] Kubernetes is an open source container orchestration engine used to automate the deployment, scaling, and management of containerized applications. Kubelet is one of the core components in Kubernetes and is responsible for managing container operations on a single node. Operator is an extension software for Kubernetes that manages applications and their components using custom resources. Apiserver is the gateway of the Kubernetes cluster. It is the central contact point accessible to all users, automation, and components in the Kubernetes cluster. Controller is a controller used to control Pods. The watch mechanism is a real-time resource status change notification mechanism based on HTTP long polling. A Pod represents a process running in the cluster. Etcd is a distributed key-value storage system used to store all cluster data in the Kubernetes cluster. Scheduler is a scheduler. Gateway is an instance of traffic processing infrastructure (such as a cloud load balancer).

[0038] In the practice of building ultra-large-scale Kubernetes clusters, the Kubelet, control plane Controller, and Operator of each node in the cluster frequently access the Kubernetes Apiserver through Watch requests to monitor resource changes, such as creating new Pods, deleting Pods, or changing Pod status. Each Watch request will occupy the computing resources and network bandwidth of the Apiserver. When the cluster node scale reaches tens of thousands and the resource objects reach millions, a large number of concurrent Watch requests will cause the Apiserver to crash due to insufficient resources or overload, causing cluster unavailability or stability issues. The current situation is as follows: Figure 1 shown.

[0039] In order to solve the existing problems in the construction of ultra-large-scale Kubernetes clusters, the present invention provides a method for optimizing Watch requests for ultra-large-scale Kubernetes clusters, which reduces the pressure on the Apiserver by reducing the number of Watch requests for accessing the Apiserver and ensures the stability and performance of the cluster.

[0040] According to an embodiment of the present invention, an embodiment of a method for optimizing Watch requests for a large-scale Kubernetes cluster is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0041] In this embodiment, a method for optimizing Watch requests for a large-scale Kubernetes cluster is provided, which can be used for the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 2 is a flowchart of a method for optimizing Watch requests in a super-large-scale Kubernetes cluster according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0042] Step S11: Generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators. The resource version number view includes the correspondence between Operators and resource version numbers.

[0043] Specifically, Figure 3 As shown in the figure, the self-developed Gateway component manages the resource objects of all Operator Watches in the cluster, generates the corresponding resource version number (ResourceVersion) view, and broadcasts the view to all Operators that watch the same resource object, so that each Operator that monitors the same resource can obtain the ResourceVersion view of other Operators. Among them, the Gateway component adapts to the gateway protocol of the Kubernetes cluster and proxies the cluster traffic.

[0044] Step S12, when the Operator performs the Watch operation, the Operator corresponding to the resource version number of the Watch resource object is searched according to the resource version number view, and a Watch request is sent to the found Operator.

[0045] Specifically, Figure 3 As shown in the figure, when the optimized Operator needs to perform a Watch operation, it will find the Operator whose ResourceVersion corresponds to the resource object it needs to Watch based on the ResourceVersion view of the resource object obtained from the Gateway, and send a Watch request to this Operator instead of sending a request to the Apiserver, thereby significantly reducing the number of Watch requests to access the Apiserver in the cluster, reducing the burden on the Apiserver, and enhancing the stability and robustness of large-scale Kubernetes clusters.

[0046] The present invention provides a method for optimizing Watch requests for ultra-large-scale Kubernetes clusters, including: generating a resource version number view in a Gateway component, and broadcasting the resource version number view to all Operators, wherein the resource version number view includes the correspondence between the Operator and the resource version number; when the Operator performs a Watch operation, the Operator whose resource version number corresponds to the Watch resource object is searched for according to the resource version number view, and a Watch request is sent to the found Operator. A self-developed Gateway component is introduced to be responsible for unified management of cluster traffic to ensure that Kubernetes cluster traffic is more evenly distributed. In addition, the running Operators are coordinated to ensure that the Operators Watching the same resource have the same ResourceVersion view. The optimized Operator receives the ResourceVersion view issued by the self-developed Gateway. When a Watch operation is required, other Operators are selected according to the view to send Watch requests, thereby reducing the number of Watch requests to access the Apiserver, alleviating the pressure on the Apiserver, and ensuring the stability and performance of the cluster.

[0047] In an optional implementation, step S11 includes the following steps:

[0048] Step S111: Use the Gateway component to manage resource objects of all Operator Watches in the cluster.

[0049] Step S112: Generate a resource version number view according to the resource object of the Operator Watch and the resource version number corresponding to the resource object.

[0050] Specifically, the Gateway component is responsible for managing the resource objects that all Operators in the cluster need to watch, and generates a resource version number view based on the resource objects that the Operator watches and the resource version numbers corresponding to the resource objects.

[0051] In an optional embodiment, the method further comprises the following steps:

[0052] Step S13: When the Operator fails to perform the Watch operation, it searches for an Operator with the same resource version number as the Watch resource object according to the resource version number view, and sends a Watch request to the found Operator.

[0053] Specifically, Figure 4As shown in the figure, when the current Watch operation fails due to network failures and you need to re-make a Watch request, select the Operator with the same ResourceVersion as the resource you want to Watch based on your saved ResourceVersion view and send a Watch request to this Operator.

[0054] In an optional embodiment, the method further comprises the following steps:

[0055] Step S14, regularly updating the resource version number view in the Gateway component.

[0056] Step S15: broadcast the updated resource version number view to all Operators.

[0057] Step S16, controlling the Operator to periodically report the resource version number of the current resource object to the Gateway component.

[0058] Specifically, the Gateway component periodically updates the ResourceVersion view of the resource object and broadcasts it to all Operators that watch the current resource, so that each Operator that monitors the same resource can obtain the ResourceVersion view of other Operators. The Operator periodically reports the ResourceVersion of the current resource object to the self-developed Gateway component, and receives and saves the ResourceVersion view of the resource object issued by the self-developed Gateway component.

[0059] In an optional implementation manner, the embodiment of the present invention may also be applied in the following practical scenarios:

[0060] (1) The present invention can make the number of nodes in a single cluster reach 10,000 and run stably, solving the problem of difficult management caused by customer service being scattered in multiple independent Kubernetes clusters, reducing cluster management costs and operation and maintenance costs, and improving the integrity and availability of the cluster.

[0061] (2) In the construction of large-scale Kubernetes clusters, the introduction of the present invention can effectively reduce the pressure on the Apiserver service, further improve the service performance, and ensure that stable service responses can still be provided under high load conditions, thereby meeting the growing business needs of customers.

[0062] In this embodiment, a super-large-scale Kubernetes cluster Watch request optimization device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0063] This embodiment provides a super-large-scale Kubernetes cluster Watch request optimization device, such as Figure 5 As shown, including:

[0064] The broadcast module 51 is used to generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators. The resource version number view includes the corresponding relationship between the Operator and the resource version number.

[0065] The query module 52 is used to search the Operator where the resource version number corresponding to the Watch resource object is located according to the resource version number view when the Operator performs the Watch operation, and send a Watch request to the found Operator.

[0066] In an optional implementation, the broadcast module 51 includes:

[0067] A management unit 511 is used to manage resource objects of all Operator Watches in the cluster using the Gateway component;

[0068] The generating unit 512 is used to generate a resource version number view according to the resource object of the Operator Watch and the resource version number corresponding to the resource object.

[0069] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0070] The ultra-large-scale Kubernetes cluster Watch request optimization device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0071] The present invention provides a super-large-scale Kubernetes cluster Watch request optimization device, which introduces a self-developed Gateway component and is responsible for unified management of cluster traffic to ensure that the Kubernetes cluster traffic is more evenly distributed. In addition, it coordinates the running Operators to ensure that the Operators that Watch the same resource have the same ResourceVersion view. The optimized Operator receives the ResourceVersion view issued by the self-developed Gateway. When a Watch operation is required, it selects other Operators to send Watch requests based on the view, reducing the number of Watch requests to access the Apiserver, alleviating the pressure on the Apiserver, and ensuring the stability and performance of the cluster.

[0072] The embodiment of the present invention also provides a computer device having the above Figure 5 The Hyperscale Kubernetes cluster Watch request optimization device is shown.

[0073] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0074] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0075] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0076] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0077] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0078] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0079] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0080] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0081] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for optimizing Watch requests in a super-large-scale Kubernetes cluster, characterized in that: The method comprises: Generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators, where the resource version number view includes the correspondence between Operators and resource version numbers; When the Operator performs a Watch operation, it searches for the Operator where the resource version number corresponding to the Watch resource object is located according to the resource version number view, and sends a Watch request to the found Operator.

2. The ultra-large-scale Kubernetes cluster Watch request optimization method according to claim 1, characterized in that: Generate a resource version number view in the Gateway component, including: Use the Gateway component to manage the resource objects of all Operator Watches in the cluster; Generate a resource version number view based on the resource object of the Operator Watch and the resource version number corresponding to the resource object.

3. The ultra-large-scale Kubernetes cluster Watch request optimization method according to claim 1, characterized in that: The method further comprises: When the Operator fails to perform the Watch operation, it searches for an Operator with the same resource version number as the Watch resource object according to the resource version number view, and sends a Watch request to the found Operator.

4. The ultra-large-scale Kubernetes cluster Watch request optimization method according to claim 1, characterized in that: The method further comprises: Regularly update the resource version number view in the Gateway component; Broadcast the updated resource version number view to all Operators.

5. The ultra-large-scale Kubernetes cluster Watch request optimization method according to claim 1, characterized in that: The method further comprises: Controls the Operator to periodically report the resource version number of the current resource object to the Gateway component.

6. The ultra-large-scale Kubernetes cluster Watch request optimization method according to claim 1, characterized in that: The Gateway component adapts to the gateway protocol of the Kubernetes cluster and proxies cluster traffic.

7. A super-large-scale Kubernetes cluster Watch request optimization device, characterized in that: The device comprises: A broadcast module, used to generate a resource version number view in the Gateway component and broadcast the resource version number view to all Operators, wherein the resource version number view includes a correspondence between Operators and resource version numbers; The query module is used to search the Operator where the resource version number corresponding to the Watch resource object is located according to the resource version number view when the Operator performs the Watch operation, and send a Watch request to the found Operator.

8. The ultra-large-scale Kubernetes cluster Watch request optimization device according to claim 7, characterized in that: The broadcast module comprises: Management unit, used to manage resource objects of all Operator Watches in the cluster using the Gateway component; The generation unit is used to generate a resource version number view according to the resource object of the Operator Watch and the resource version number corresponding to the resource object.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the ultra-large-scale Kubernetes cluster Watch request optimization method according to any one of claims 1 to 6 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the ultra-large-scale Kubernetes cluster Watch request optimization method described in any one of claims 1 to 6.

Images