A method for real-time dynamic management of kubernetes virtual nodes provider

Abstract

The application discloses a kind of real-time dynamic management kubernetes virtual node provider's method, including component console opening serverless k8s cluster, using customized scheduler, see a vk node start and register in serverless k8s cluster if it meets expectation, start provider service.The application changes the state of provider pod by manual triggering, observes the data production of kafka, and when vk consumes change message, the corresponding pod in k8s can be found according to the unique identifier, and the change of provider pod is synchronized to the pod in k8s through protocol conversion, which realizes real-time synchronization of state from provider pod to k8s pod, avoids performance problems of provider caused by frequent polling, realizes data synchronization by message queue, and the architecture is more reasonable, and will not cause performance overload of provider, so that risk can be controlled.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of kubernetes cloud platform, and particularly relates to a method for real-time dynamic management of a kubernetes virtual node provider. BACKGROUND

[0002] In a cloud native scenario, there is an application scenario of extending kubelet through a virtual node (virtual kubelet, hereinafter referred to as vk). In a serverless k8s cluster, there is no concept of kubelet and node, so it is necessary to register to the k8s apiserver as kubelet internally and access each provider (a provider maintains a set of interfaces of a pod life cycle, which can be understood as providing different implementation manners for managing k8s pods) provided by a cloud manufacturer externally through the vk.

[0003] In the general solution under this business scenario, although the existing vk source code provides the implementation of synchronizing state changes from the k8s apiserver side to the provider, it does not provide a way for the provider side to synchronize state changes to the k8s apiserver. At this time, the existing method requires the developer to periodically poll the provider related interface in the vk to obtain the latest pod state. When the number of pods is huge, it will cause the load pressure of the provider to be too large. Moreover, only one vk is registered in the entire serverless cluster, the vk may need to manage tens of thousands of pods, which is a single point service and lacks a dynamic expansion mechanism, and there will be a large load pressure and security performance problem. SUMMARY

[0004] This part aims to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this part and the abstract and title of the specification of the application to avoid obscuring the purpose of this part, the abstract and the title of the specification. Such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] In view of the above existing problems, the present application is proposed.

[0006] Therefore, the present application aims to provide a method for real-time dynamic management of kubernetes virtual node provider, which aims to solve the problem of "in this business scenario, the existing vk source code provides the implementation of synchronizing state changes from the k8s apiserver side to the provider, but does not provide a way for the provider side to synchronize state changes to the k8s apiserver. At this time, the existing method requires the developer to periodically poll the provider-related interface in vk to obtain the latest pod state. When the number of pods is huge, it will cause the provider load to be too large, and the entire serverless cluster only registers one vk, which may need to manage thousands of pods, which is a single-point service and lacks a dynamic expansion mechanism, causing large load pressure and security performance problems."

[0007] To solve the above technical problems, the present application provides the following technical solutions:

[0008] A method for real-time dynamic management of kubernetes virtual node provider, comprising the following steps:

[0009] SI, the component console opens a serverless k8s cluster, uses a custom scheduler, and sees a vk node start and register in the serverless k8s cluster if the expected result is met, and starts the provider service;

[0010] S2, manually trigger the state change of the provider pod, and observe the data production of kafka. When the vk consumes the change message, it can find the corresponding pod in k8s according to the unique identifier, and through protocol conversion, it can realize the synchronization of the provider pod change to the k8s pod.

[0011] S3, create n pods in the serverless k8s cluster, and observe the writing of the corresponding data in elasticsearch. Continuously create pods until the threshold value configured in the configmap is reached. During this process, the vk periodically queries the es to determine whether the number of pods managed by itself reaches the threshold value. After reaching the threshold value, the vk creates a cr in the kubernetes cluster to trigger the vk node expansion.

[0012] S4, repeat S3 several times to observe the expansion operation.

[0013] S5, after capacity expansion, continuously create pods in the serverless k8s cluster, verify that the pods are scheduled to the newly expanded vk nodes, observe that the node-managed pods are running, the pod information is stringed to other vk nodes, the number of pods managed by each follower reaches the threshold, and a cr is created in the k8s cluster, and the vk-operator creates a new vk node through the watch to the creation of the cr;

[0014] S6, after the implementation is completed, unsubscribe from the serverless k8s cluster, kill the provider service, and clear the resources on the kubernetes cluster.

[0015] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, wherein: there are n vks in the same consumer group, which ensures that the messages are not consumed repeatedly after capacity expansion.

[0016] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, wherein: all the vk nodes can be consumed, all the vk nodes can access the k8s cluster, the consumer group limits the messages to be consumed only once, the provider pod is uniquely associated with the k8s pod, and any vk can update the state of the k8s pod.

[0017] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, wherein: the vk-operator is deployed in the k8s cluster, and the function of the vk-operator is to process the creation of the vk node.

[0018] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, wherein: the k8s cluster is a management cluster, and is not a user's own business cluster.

[0019] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, wherein: the number of vk nodes is kept odd, an election mechanism is introduced, only one vk exists at the initialization time, and the vk automatically becomes a leader at this time; when there are multiple vk nodes, a leader is generated through a paxos or raft protocol election, and the rest are followers.

[0020] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, the single vk node creates and maintains a provider pod, and the provider pod is stored in elasticsearch.

[0021] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, the total number of the vk nodes after creation is kept odd, and this is controlled by the spec in the crd, and the leader has the number information of the current vk node every time the creation is performed.

[0022] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, the pod node scheduling is realized by extending the kube-scheduler through a custom scheduler.

[0023] As a preferred scheme of the method for real-time dynamic management of the kubernetes virtual node provider, the threshold of the number of the pods is configured in the management cluster in the form of a configmap, the custom scheduler implements a priority scheduling strategy according to the threshold of the number of the pods managed by each vk node, and the created k8s pod is preferentially scheduled to the vk whose number of pods does not reach the threshold.

[0024] The present application has the following beneficial effects:

[0025] 1. The state change of the provider pod is manually triggered, the data production of kafka is observed, when the vk consumes the change message, the corresponding pod can be found in k8s according to the unique identifier, the protocol conversion is performed, the change of the provider pod is synchronized to the pod in k8s, the real-time synchronization of the state from the provider pod to the k8s pod is realized, the performance problem of the provider caused by frequent polling is avoided, the data synchronization is realized through the message queue, the architecture is more reasonable, and the performance overload of the provider is avoided, so that the risk is controllable.

[0026] 2. By creating n pods in a serverless Kubernetes cluster and observing the write activity of corresponding data in Elasticsearch, pods are continuously created until the threshold configured in the configmap is reached. This implements a mechanism for dynamically scaling VK nodes. Each VK node manages only its own set of pods, avoiding the single point of failure issue of VK. Introducing Elasticsearch as storage provides efficient insertion and querying, and a custom scheduler ensures that the number of pods managed by each VK is dynamically balanced. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0028] Figure 1 This is a system architecture diagram of a method for real-time dynamic management of Kubernetes virtual node providers according to the present invention. Detailed Implementation

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0031] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0032] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0033] Reference Figure 1This invention provides a method for real-time dynamic management of Kubernetes virtual node providers, comprising the following steps:

[0034] SI. The component console opens a serverless Kubernetes cluster. Using a custom scheduler, if the expected result is achieved, a VK node will start and register with the serverless Kubernetes cluster, start the provider service, synchronize the provider status to the VK, introduce a message middleware similar to Kafka, and label the VK to which the pod belongs for each pod scheduled to the VK.

[0035] S2. Manually trigger the state change of the provider pod and observe the data production status of Kafka. When the VK consumes the change message, it can find the corresponding pod in Kubernetes based on this unique identifier and realize the synchronization of the provider pod change to the Kubernetes pod through protocol conversion.

[0036] S3. Create n pods in a serverless Kubernetes cluster, observe the writing of corresponding data in Elasticsearch, continuously create pods, and synchronize data to Elasticsearch when the provider pod is in production, until the threshold configured in the configmap is reached. During this process, the VK periodically queries Elasticsearch to determine if the number of pods it manages has reached the threshold. After reaching the threshold, the VK creates a cr in the Kubernetes cluster to trigger the expansion of the VK node.

[0037] S4. Repeat S3 several times and observe the expansion operation.

[0038] S5. After scaling up, continuously create pods in the serverless Kubernetes cluster to verify that the pods are scheduled to the newly scaled VK nodes. Observe the pods managed by the nodes running, and pod information is linked to other VK nodes. When the number of pods managed by each follower reaches the threshold, create a cr in the Kubernetes cluster. After the VK-operator watches the creation of the cr through tuning, it creates a new VK node. The new node will automatically join the VK node group as a follower. Each VK node maintains the provider pods it manages through Elasticsearch. The interaction process between VK and provider will only query and modify the provider pods it manages.

[0039] S6. After implementation, unsubscribe from the serverless Kubernetes cluster, kill the provider service, and remove resources from the Kubernetes cluster.

[0040] Within the same consumer group, there are n virtual key (VK) instances to ensure that messages are not consumed repeatedly after scaling up, so that each change to the provider is consumed by only one of the VK instances.

[0041] Furthermore, all VK nodes can consume messages, all VK nodes can access the Kubernetes cluster, the consumer group restricts messages to be consumed only once, the provider pod is uniquely associated with the Kubernetes pod, any VK can update the state of the Kubernetes pod, so there is no restriction that the provider pod can only be consumed by the VK it manages, and all VK nodes can consume messages.

[0042] Furthermore, vk-operator is deployed in the Kubernetes cluster. The role of vk-operator is to handle the creation of vk nodes.

[0043] Furthermore, the Kubernetes cluster mentioned is a management cluster, not the user's own business cluster.

[0044] Furthermore, the number of VK nodes is kept odd, and an election mechanism is introduced. Initially, there is only one VK, which automatically becomes the leader. When there are multiple VK nodes, a leader is elected through a Paxos-like or Raft protocol, and the rest are followers.

[0045] Furthermore, the provider pods created and maintained by a single VK node are stored in Elasticsearch.

[0046] Furthermore, the total number of VK nodes must remain odd after creation. This is controlled by the spec in the CRD. Each time a VK node is created, the leader has the current number of VK nodes, and new nodes automatically join the VK node group as followers. Each VK node maintains its managed provider pods through Elasticsearch, and the interaction between the VK and the provider only involves querying and modifying the provider pods it manages.

[0047] Furthermore, the pod node scheduling is achieved by extending kube-scheduler with a custom scheduler.

[0048] Furthermore, the threshold for the number of pods is configured in the management cluster in the form of a configmap. The custom scheduler implements a priority scheduling strategy based on this threshold of "the number of pods managed by each VK node", prioritizing the scheduling of created Kubernetes pods to VKs whose number of pods has not reached the threshold.

[0049] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for real-time dynamic management of Kubernetes virtual node providers, characterized in that: Includes the following steps: SI: Activate the serverless Kubernetes cluster in the component console. Using a custom scheduler, if all goes as expected, you will see a VK node start and register to the serverless Kubernetes cluster, and the provider service start. All VK nodes can consume data, all VK nodes can access the Kubernetes cluster, the consumer group restricts messages to be consumed only once, each provider pod is uniquely associated with a Kubernetes pod, and any VK can update the state of the Kubernetes pod. S2. Manually trigger the state change of the provider pod and observe the data production status of Kafka. When the VK consumes the change message, it can find the corresponding pod in Kubernetes based on this unique identifier. Through protocol conversion, the change of the provider pod is synchronized to the Kubernetes pod. S3. Create n pods in a serverless Kubernetes cluster, observe the write status of the corresponding data in Elasticsearch, and continuously create pods until the threshold configured in the configmap is reached. During this process, VK periodically queries Elasticsearch to determine if the number of pods it manages has reached the threshold. Once the threshold is reached, VK creates CR in the Kubernetes cluster to trigger VK node expansion. S4. Repeat S3 several times and observe the expansion operation. The number of VK nodes is kept odd. An election mechanism is introduced. Initially, there is only one VK, which automatically becomes the leader. When there are multiple VK nodes, a leader is elected through a Paxos-like or Raft protocol, and the rest are followers. S5. After scaling up, continue to create pods in the serverless Kubernetes cluster to verify that the pods are scheduled to the newly scaled VK nodes. Observe the pods managed by the nodes running, and pod information is linked to other VK nodes. When the number of pods managed by each follower reaches the threshold, create a cr in the Kubernetes cluster. After the vk-operator watches the creation of the cr through tuning, create a new VK node. S6. After implementation, unsubscribe from the serverless Kubernetes cluster, kill the provider service, and remove resources from the Kubernetes cluster.

2. The method for real-time dynamic management of Kubernetes virtual node providers according to claim 1, characterized in that: Within the same consumer group, there are agreed to be n vk, ensuring that messages are not consumed repeatedly after expansion.

3. The method for real-time dynamic management of Kubernetes virtual node providers according to claim 1, characterized in that: The Kubernetes cluster deploys vk-operator, whose role is to handle the creation of vk nodes.

4. The method for real-time dynamic management of Kubernetes virtual node providers according to claim 1, characterized in that: The Kubernetes cluster mentioned is a management cluster, not the user's own business cluster.

5. The method for real-time dynamic management of Kubernetes virtual node providers according to claim 1, characterized in that: The provider pods created and maintained by a single VK node are stored in Elasticsearch.

6. The method for real-time dynamic management of Kubernetes virtual node providers according to claim 1, characterized in that: The total number of VK nodes must remain odd after creation. This is controlled by the spec in the CRD. Each time a VK node is created, the leader has the current number of VK nodes.

7. The method for real-time dynamic management of Kubernetes virtual node providers according to claim 1, characterized in that: Pod node scheduling is achieved by extending kube-scheduler with a custom scheduler.

8. The method for real-time dynamic management of Kubernetes virtual node providers according to claim 1, characterized in that: The threshold for the number of pods is configured in the management cluster in the form of a configmap. The custom scheduler implements a priority scheduling policy based on this threshold of "the number of pods managed by each VK node", giving priority to scheduling the created Kubernetes pods to VKs whose number of pods has not reached the threshold.

Images