A Kubernetes pod network resource adaptive control method

Abstract

The application provides a Kubernetes pod network resource adaptive control method, and belongs to the technical field of pod network resource adaptive control, and comprises the following steps: step S1, all existing pods are traversed, and a network priority is set for each pod; step S2, the cgroup net_cls of the pod and the container is set; step S3, for each container, the container network namespace is entered, and bandwidth management is enabled; step S4, a new pod is identified and bandwidth setting is triggered; step S5, whether the cgroup configuration is effective is checked; step S6, bandwidth adaptive control is performed on all containers; and step S7, the network bandwidth of offline services is limited through the Linux EDT mechanism, and network bandwidth control is realized. The application realizes adaptive control of pod network resources.

Description

Technical Field

[0001] This invention relates to Pod network resources, and more particularly to an adaptive control method for Kubernetes Pod network resources, belonging to the field of adaptive control technology for Pod network resources. Background Technology

[0002] With the widespread adoption of cloud computing and microservice architectures, Kubernetes has become the de facto standard in container orchestration. In the complex distributed environments it manages, multiple application pods typically share the same physical network resources on the same node, leading to significant network service quality (QoS) challenges, particularly in terms of network bandwidth contention. Existing solutions have the following drawbacks:

[0003] Limitations of Static Configuration: Traditional Kubernetes network resource management (such as setting bandwidth via requests and limits) is typically static and rigid. Once set, it cannot be dynamically adjusted based on real-time network conditions. This leads to two common problems: 1) Resource Waste: Bandwidth reserved for low-priority Pods cannot be used by high-priority services when idle, resulting in idle resources. 2) Sudden Traffic Surges: When high-priority services experience sudden traffic surges, low-priority services still occupy fixed bandwidth and cannot yield, potentially causing delays, jitter, or even interruptions in critical services (such as online transactions and real-time audio / video), severely impacting user experience and system stability.

[0004] Lack of granular priority control: Standard Kubernetes network policies focus more on security isolation (NetworkPolicy) than on bandwidth priority management based on service type. The system cannot automatically identify and differentiate between Pods with different service characteristics, such as "online" (latency-sensitive) and "offline" (batch processing, latency-tolerant), and provide them with differentiated network treatment.

[0005] Complexity of fault recovery: When the network QoS management module (such as the net QoS module of this invention) restarts or a Pod is rebuilt, existing bandwidth control policies may be lost. Traditional methods lack an automated recovery mechanism to recapture all existing Pods and reconfigure application policies for them, resulting in gaps in policy execution and an uncontrolled network state.

[0006] Insufficient control precision: Simple bandwidth limiting (such as using TC's HTB limiting) cannot achieve microsecond-level precise traffic shaping, making it difficult to meet the stringent requirements of modern cloud-native applications for low latency and high throughput. Summary of the Invention

[0007] To address the aforementioned issues, this invention provides a Kubernetes pod network resource adaptive control method to achieve adaptive control and adjustment of pod network resources.

[0008] To achieve the above objectives, the technical solution of the present invention is: a Kubernetes pod network resource adaptive control method, comprising the following steps:

[0009] Step S1: Iterate through all existing Pods and set the network priority for each Pod;

[0010] Step S2: Configure the cgroup net_cls of the Pod and container, and write the network priority identifier set in step S1 into the cgroup net_cls controller of the Pod and all its containers;

[0011] Step S3: For each container, enter the container network namespace and enable bandwidth management;

[0012] Step S4: Identify the new Pod and trigger bandwidth settings;

[0013] Step S5: Verify whether the cgroup configuration is effective;

[0014] Step S6: Perform adaptive bandwidth control for all containers;

[0015] Step S7: Limit the offline service network bandwidth through the Linux EDT mechanism to achieve network bandwidth control.

[0016] Furthermore, in step S1, the list of all existing Pods in the current system is obtained through viewer.ListPodsWithOptions(), and then the network priority is set for each Pod by determining whether it is online.

[0017] Furthermore, when a Pod is online, value = 0, indicating normal priority; if a Pod is offline, value = -1, indicating special priority.

[0018] Furthermore, step S3 specifically includes the following steps: reading the cgroup.procs file of the container cgroup, obtaining the PID of each container master process, using the command nsenter to enter the container network namespace, using the Linux TC module to enable bandwidth management for the network stack inside the container, and setting the bandwidth and waterline in the eBPF maps data structure, where bandwidth is divided into low and high.

[0019] Furthermore, step S4 specifically includes the following steps: when a new Pod is detected by the Kubernetes Informer event mechanism, network priority and bandwidth are set for each container through cgroup net_cls, and bandwidth management settings are enabled in the container network namespace using the linuxTC module.

[0020] Furthermore, step S6 specifically includes the following steps:

[0021] S6-1 mounts an eBPF program in the Linux Traffic Control subsystem for bandwidth management;

[0022] S6-2 reads the system settings for bandwidth and waterline, and sets the bandwidth rate, with an initial value of bandwidth.low;

[0023] Step S6-3: Obtain the priority prio through cgroup net_cls.classid;

[0024] Step S6-4: Adaptively adjust the bandwidth rate based on prio. If prio is 0, it indicates online service, and the bandwidth rate remains unchanged; if prio is -1, it indicates offline service, and the bandwidth rate is adjusted adaptively.

[0025] Furthermore, the logic for adaptively adjusting the bandwidth rate in step S6-4 includes the following:

[0026] The actual online bandwidth (rate_past) and actual offline bandwidth (offline_rate_past) within the statistical period are calculated, and then a linear interpolation method is used to adaptively adjust the offline service bandwidth.

[0027] Furthermore, linear interpolation specifically includes the following:

[0028] If the actual online bandwidth rate_past is greater than the waterline, the offline bandwidth will decrease linearly in proportion to the value delta that exceeds the waterline, and will not fall below low_rate. If the actual online bandwidth rate_past is less than the waterline, the offline bandwidth will increase linearly in proportion to the value delta that is less than the waterline, and will not rise above high_rate.

[0029] Furthermore, step S7 specifically includes the following steps:

[0030] Step S7-1: Count offline service traffic;

[0031] Step S7-2: Obtain the current time;

[0032] Step S7-3: Obtain the planned sending time of the data packet;

[0033] Step S7-4: Calculate the delay time for this data packet, which is inversely proportional to the rate mentioned above;

[0034] Step S7-5: Calculate the next sending time;

[0035] Step S7-6: Copy the next sending time to skb->tstamp to delay the sending of data packets.

[0036] The beneficial effects of the Kubernetes pod network resource adaptive control method of the present invention are as follows:

[0037] This invention enables automated discovery and configuration, automatically discovering all Pods in the cluster (including newly created and existing ones) and configuring application network QoS policies for them without manual intervention, ensuring the comprehensiveness and persistence of the policies.

[0038] This invention can intelligently differentiate based on priority, and can automatically set different network priorities (such as classid) according to the business attributes of Pods (such as online / offline), laying the foundation for subsequent differentiated scheduling.

[0039] This invention can dynamically and adaptively adjust and can sense network traffic conditions in real time. When the traffic of high-priority services is lower than a preset level, low-priority services are allowed to make full use of idle bandwidth. When high-priority services need more resources, bandwidth can be intelligently and smoothly reclaimed, thereby maximizing overall resource utilization while ensuring the SLA of critical services.

[0040] This invention utilizes the high-performance control of eBPF and TC. Specifically, it leverages the kernel programmability of the eBPF program and the powerful traffic shaping capabilities of the Linux Traffic Control (TC) subsystem to achieve fine-grained management with user-mode awareness and kernel-mode execution. In particular, it achieves precise bandwidth control through microsecond-level delay scheduling via the EDT (Earliest Departure Time) mechanism. Detailed Implementation

[0041] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0042] This invention discloses an adaptive control method for Kubernetes pod network resources. This method is based on a deep integration of cgroups, eBPF, and TC to achieve intelligent, dynamic, and fine-grained allocation of network bandwidth in a cloud-native environment, ultimately ensuring the service quality of critical businesses and improving the overall network efficiency of the cluster. The method is described in detail below with reference to specific embodiments.

[0043] Example 1

[0044] A method for adaptive control of Kubernetes pod network resources includes the following steps:

[0045] Step S1 involves iterating through all existing Pods and setting a network priority for each Pod. This step automates the process by iterating through all Pods (including historical and newly created ones) and dynamically assigning different network priority identifiers based on the Pod's online / offline status. Specifically, this includes:

[0046] The `viewer.ListPodsWithOptions()` method retrieves a list of all existing Pods in the current system. This step ensures that the net QoS module does not miss any bandwidth and priority settings for historical Pods when it starts or restarts. Then, for each Pod, it checks whether it is online. If it is online, its value = 0, indicating normal priority; if it is offline, its value = -1, indicating special priority (such as isolation or rate limiting).

[0047] Step S2 configures the cgroup net_cls for Pods and containers. This step writes the priority identifier (classid) to the cgroup net_cls controller of the Pod and all its containers. This allows the Linux kernel's flow control (TC) subsystem to automatically identify and differentiate traffic of different priorities based on this classid, laying the foundation for subsequent differentiated scheduling. Specifically, this includes:

[0048] By using cgroup net_cls, the classid is written to the cgroup file net_cls.classid of the Pod and all its containers, thereby enabling traffic grouping and isolation in the following steps; the cgroup file net_cls.classid is determined according to the priority (i.e., value) set in step S1.

[0049] Step S3: For each container, enter the container network namespace and enable bandwidth management, specifically including:

[0050] Read the cgroup.procs file of the container cgroup to obtain the PID of each container's main process. Use the nsenter command to enter the container's network namespace. Use the Linux TC module to enable bandwidth management for the network stack inside the container and set the bandwidth and waterline in the eBPF maps data structure, where bandwidth is divided into low and high.

[0051] Step S4, identify the new Pod and trigger bandwidth settings, specifically including:

[0052] When a new Pod is detected by the Kubernetes Informer event mechanism, network priority (refer to step S1) and bandwidth (refer to step S3) are set for each container using cgroupnet_cls, and bandwidth management settings are enabled in the container network namespace using the Linux TC module.

[0053] Step S5, verify whether the cgroup configuration is effective, specifically including:

[0054] Check that the cgroup configuration is written correctly and ensure that the bandwidth policy is in effect. The cgroup configuration here includes the bandwidth configuration in net_cls.classid and the eBPF maps data structure mentioned in step S2.

[0055] Step S6 involves performing adaptive bandwidth control for all containers, specifically including:

[0056] S6-1 mounts an eBPF program in the Linux Traffic Control (TC) subsystem for bandwidth management.

[0057] S6-2, Read the bandwidth and waterline from the eBPF maps data structure set in step S3, set the bandwidth rate with an initial value of bandwidth.low, and set the waterline to the waterline set in step S3.

[0058] Step S6-3: Obtain the priority prio using cgroup net_cls.classid. Here, prio, like classid and value in the previous steps, represents priority; the different representations are to distinguish different setup stages.

[0059] Step S6-4: Adaptively adjust the bandwidth rate based on prio. If prio is 0, it indicates online service, and the bandwidth rate remains unchanged; if prio is -1, it indicates offline service, and the bandwidth rate is adjusted adaptively.

[0060] Step S7, Network Bandwidth Control. This step involves the network bandwidth control logic within the Linux kernel. The bandwidth settings from the previous steps ultimately take effect through this step, specifically by limiting offline service network bandwidth through the Linux EDT (Earliest Departure Time) mechanism. The specific steps are as follows:

[0061] Step S7-1: Count offline service traffic;

[0062] Step S7-2: Obtain the current time;

[0063] Step S7-3: Obtain the planned sending time of the data packet;

[0064] Step S7-4: Calculate the delay time for this data packet, which is inversely proportional to the rate mentioned above;

[0065] Step S7-5: Calculate the next sending time;

[0066] Step S7-6: Copy the next sending time to skb->tstamp to delay the sending of data packets.

[0067] This step implements microsecond-level precise traffic shaping based on the Linux EDT mechanism. Specifically:

[0068] Bandwidth limiting is achieved using EDT (Earliest Departure Time) instead of the traditional token bucket algorithm. By calculating the "next send time" (skb->tstamp) for each data packet and introducing a microsecond-level delay, the transmission rate is controlled. This method is more precise, reduces bursts, and thus achieves smoother traffic and lower latency. This is the technical guarantee for achieving high-performance, high-precision bandwidth control, and is especially suitable for latency-sensitive cloud-native environments.

[0069] Example 2

[0070] This embodiment, based on embodiment 1, further details the logic of adaptively adjusting the bandwidth rate in step S6. Specifically, after step S6-4, it also includes the following:

[0071] The actual online bandwidth rate_past and the actual offline bandwidth offline_past within the statistical period.

[0072] A linear interpolation method is used to adaptively adjust offline service bandwidth. If the actual online bandwidth rate_past is greater than the waterline, the offline bandwidth is linearly reduced proportionally to the value delta (i.e., rate_past - waterline), and will not fall below low_rate. If the actual online bandwidth rate_past is less than the waterline, the offline bandwidth is linearly increased proportionally to the value delta (i.e., waterline - rate_past), and will not fall above high_rate.

[0073] This step implements a bandwidth adaptive feedback control loop that integrates eBPF and TC. It introduces a dynamic adaptive adjustment algorithm, rather than simply static rate limiting. This algorithm uses a waterline as a reference and adjusts the bandwidth of low-priority (offline) services in real time using linear interpolation. Specifically:

[0074] Monitoring: The eBPF program is responsible for real-time statistics of actual online bandwidth (rate_past).

[0075] Decision: Compare rate_past with the preset waterline and calculate the difference delta.

[0076] Adjustment: Based on the delta value, linearly increase or decrease the bandwidth (rate) of offline services and limit it within the range of [low_rate, high_rate].

[0077] Value: This closed-loop control mechanism ensures that low-priority services can make full use of idle bandwidth when high-priority services are not busy (improving efficiency); and that bandwidth can be quickly and smoothly reclaimed when high-priority services require resources (ensuring SLA).

[0078] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

Claims

1. A method for adaptive control of Kubernetes pod network resources, characterized in that, Includes the following steps: Step S1: Iterate through all existing Pods and set the network priority for each Pod; Step S2: Configure the cgroup net_cls of the Pod and container, and write the network priority identifier set in step S1 into the cgroup net_cls controller of the Pod and all its containers; Step S3: For each container, enter the container network namespace and enable bandwidth management; Step S4: Identify the new Pod and trigger bandwidth settings; Step S5: Verify whether the cgroup configuration is effective; Step S6: Perform adaptive bandwidth control for all containers; Step S7: Limit the offline service network bandwidth through the Linux EDT mechanism to achieve network bandwidth control.

2. The method according to claim 1, characterized in that, In step S1, the viewer.ListPodsWithOptions() is used to obtain a list of all existing Pods in the current system, and then the network priority is set for each Pod by determining whether it is online.

3. The method according to claim 2, characterized in that, When a Pod is online, value = 0, indicating normal priority; if a Pod is offline, value = -1, indicating special priority.

4. The method according to claim 1, characterized in that, Step S3 specifically includes the following steps: read the cgroup.procs file of the container cgroup, obtain the PID of each container master process, enter the container network namespace using the nsenter command, enable bandwidth management for the network stack inside the container using the Linux TC module, and set the bandwidth and waterline in the eBPF maps data structure, where bandwidth is divided into low and high.

5. The method according to claim 1, characterized in that, Step S4 specifically includes the following steps: When a new Pod is detected by the Kubernetes Informer event mechanism, network priority and bandwidth are set for each container through cgroup net_cls, and bandwidth management settings are enabled in the container network namespace using the Linux TC module.

6. The method according to claim 1, characterized in that, Step S6 specifically includes the following steps: S6-1 mounts an eBPF program in the Linux Traffic Control subsystem for bandwidth management; S6-2 reads the system settings for bandwidth and waterline, and sets the bandwidth rate, with an initial value of bandwidth.low; Step S6-3: Obtain the priority prio through cgroup net_cls.classid; Step S6-4: Adaptively adjust the bandwidth rate based on prio. If prio is 0, it indicates online service, and the bandwidth rate remains unchanged; if prio is -1, it indicates offline service, and the bandwidth rate is adjusted adaptively.

7. The method according to claim 6, characterized in that, The logic for adaptively adjusting the bandwidth rate in step S6-4 includes the following: The actual online bandwidth (rate_past) and actual offline bandwidth (offline_rate_past) within the statistical period are calculated, and then a linear interpolation method is used to adaptively adjust the offline service bandwidth.

8. The method according to claim 7, characterized in that, Linear interpolation methods specifically include the following: If the actual online bandwidth rate_past is greater than the waterline, the offline bandwidth will decrease linearly in proportion to the value delta that exceeds the waterline, and will not fall below low_rate. If the actual online bandwidth rate_past is less than the waterline, the offline bandwidth will increase linearly in proportion to the value delta that is less than the waterline, and will not rise above high_rate.

9. The method according to claim 1, characterized in that, Step S7 specifically includes the following steps: Step S7-1: Count offline service traffic; Step S7-2: Obtain the current time; Step S7-3: Obtain the planned sending time of the data packet; Step S7-4: Calculate the delay time for this data packet, which is inversely proportional to the rate mentioned above; Step S7-5: Calculate the next sending time; Step S7-6: Copy the next sending time to skb->tstamp to delay the sending of data packets.