Container resource limiting method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention is applicable to the technical field of computers, and provides a container resource limiting method and device, electronic equipment and a storage medium, and the container resource limiting method is applied to a container arrangement platform and comprises the following steps: adding a setting configuration item for a data volume in a configuration file of a container group, the set configuration item is used for limiting read-write resources of the data volume by the containers in the container group; the data volume is mounted in the container of the container group; analyzing and setting the set configuration item based on the data volume plug-in; the analysis and configuration actions are not executed through a standard interface of the container arrangement platform.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a container resource restriction method, device, electronic device and storage medium. Background Art

[0002] In the container orchestration platform (Kubernetes, K8s), since cluster resources are shared, resource preemption is prone to occur. For the central processing unit (CPU) and memory, resources can be reserved and isolated through the existing resource quotas of K8s. For input / output (IO) resources, K8s lacks an effective reservation and isolation mechanism, resulting in some Pods (container groups) preempting too many IO resources, and some Pods are IO starved.

[0003] Currently, developers implement IO resource restrictions by modifying the CRI (ContainerRuntimeInterface, a standard interface provided by K8s that defines the interaction between Kubernetes and the container runtime. Under the native interface definition, CPU and memory can be restricted). With the continuous iteration of K8s, CRI is also continuously updated, which makes the interface definition modifications made by developers in the current version prone to failure and non-portability in the next version, poor compatibility, and low development efficiency. Therefore, how to improve the compatibility of the IO resource restriction solution of the K8s platform has become an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above problems, embodiments of the present invention provide a container resource restriction method, device, electronic device and storage medium.

[0005] The technical solution of the present invention is achieved in this way:

[0006] In one aspect, an embodiment of the present invention provides a container resource restriction method, which is applied to a container orchestration platform. The method includes:

[0007] Adding a configuration item for the data volume in a configuration file of the container group, wherein the configuration item is used for the container in the container group to restrict the read and write resources of the data volume; the data volume is mounted in the container of the container group;

[0008] The configuration items are parsed and set based on the data volume plug-in; and the parsing and configuration actions are not performed through a standard interface of the container orchestration platform.

[0009] In the above solution, the parsing and setting of the configuration items based on the data volume plug-in includes:

[0010] Parsing the setting configuration item to obtain quality of service QoS configuration information in the setting configuration item;

[0011] Generate resource configuration information of the control group of the container group based on the QoS configuration information;

[0012] Based on the resource configuration information, the control group is updated.

[0013] In the above solution, parsing the setting configuration item to obtain the QoS configuration information in the setting configuration item includes:

[0014] Determine whether QoS configuration information is configured in the setting configuration item;

[0015] If QoS configuration information is configured in the setting configuration item, then checking whether the volume type of the data volume is a valid type;

[0016] If the volume type of the data volume is a valid type, obtaining the major and minor device numbers of the block device where the data volume is located;

[0017] If the primary and secondary device numbers are normal, the configuration item is parsed successfully and the QoS configuration information is obtained.

[0018] In the above solution, updating the control group based on the resource configuration information includes:

[0019] Obtain a unique identifier of a control group of the container group;

[0020] Determining path information of each subsystem of the control group based on a unique identifier of the control group;

[0021] Based on the path information and the resource configuration information, parameters of each subsystem of the control group are set.

[0022] In the above solution, the parsing and setting of the configuration items based on the data volume plug-in includes:

[0023] Determining the type of the data volume;

[0024] The setting configuration item is parsed and set based on the data volume plug-in corresponding to the type of the data volume.

[0025] In the above solution, after parsing and setting the configuration item based on the data volume plug-in, the method further includes:

[0026] Adding process identifiers of containers in the container group to a subsystem of a control group of the container group.

[0027] In the above solution, adding the process identifier of the container in the container group to the subsystem of the control group of the container group includes:

[0028] Obtain a unique identifier of a control group of the container group;

[0029] Determining path information of each subsystem of the control group based on a unique identifier of the control group;

[0030] Based on the path information, process identifiers of containers in the container group are added to corresponding subsystems.

[0031] On the other hand, an embodiment of the present application further provides a container resource restriction device, including:

[0032] An adding module, used for adding a setting configuration item for a data volume in a configuration file of a container group, wherein the setting configuration item is used for a container in the container group to restrict the read and write resources of the data volume; the data volume is mounted in a container of the container group;

[0033] A parsing and setting module is used to parse and set the configuration items based on the data volume plug-in; the parsing and configuration actions are not performed through the standard interface of the container orchestration platform.

[0034] On the other hand, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the container resource limitation method in the above solution.

[0035] On the other hand, an embodiment of the present invention provides an electronic device, including a processor and a memory, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the steps of the container resource limitation method provided in the first aspect of the embodiment of the present invention.

[0036] In another aspect, an embodiment of the present invention provides a computer-readable storage medium, including: the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the container resource restriction method provided in the first aspect of the embodiment of the present invention are implemented.

[0037] The embodiment of the present application adds a setting configuration item for the data volume in the configuration file of the container group. The setting configuration item is used for the containers in the container group to restrict the reading and writing resources of the data volume, and the data volume is mounted in the container of the container group. The setting configuration item is parsed and set based on the data volume plug-in, and the parsing and configuration actions are not performed through the standard interface of the container orchestration platform. The embodiment of the present application adds a setting configuration item to the configuration file of the container group, and parses and sets the setting configuration item through the data volume plug-in, so as to restrict the reading and writing resources of the data volume for the containers in the container group. It does not need to rely on the standard interface of the container orchestration platform, does not need to modify the interface definition, only needs to make simple changes, and can be quickly ported to different versions of the container orchestration platform, with wide versatility and compatibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of an implementation flow of a container resource restriction method provided by an embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of a Pod data volume QoS architecture provided by an embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of a configuration item parsing process provided by an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of a QoS setting process provided by an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of a QoS validation process provided by an embodiment of the present invention;

[0043] Figure 6 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] In a Kubernetes cluster, Pod (container group) is the basis of all business types and the smallest unit level managed by K8s. It is a combination of one or more containers. One or more containers can run in a Pod, and the containers share the same Pod's network environment. K8s uses Pod as the smallest unit for scheduling, expansion, resource sharing, and lifecycle management.

[0046] In a Kubernetes cluster, for CPU and memory, resources can be reserved and isolated through K8s' existing resource quotas. For input / output (IO) resources, K8s lacks an effective reservation and isolation mechanism, resulting in some Pods (container groups) occupying too many IO resources and some Pods experiencing IO starvation.

[0047] The standard interfaces provided by K8s include but are not limited to CNI (Container Network Interface), CSI (Container Storage Interface, CRI, etc. Among them, CRI is a standard interface provided by K8s, which is used to define the interaction between Kubernetes and container runtime. Under the native interface definition, CPU and memory can be restricted.

[0048] Developers will implement IO resource restrictions by modifying the interface definition of CRI. As K8s continues to iterate, CRI is also continuously updated, which makes the interface definition modifications made by developers in the current version vulnerable to failure and non-portability in the next version, resulting in poor compatibility and low development efficiency. Therefore, how to improve the compatibility of the IO resource restriction solution of the K8s platform has become an urgent problem to be solved.

[0049] In view of the shortcomings of the above-mentioned related technologies, an embodiment of the present invention provides a container resource restriction method, which can implement container IO resource restriction without using CRI. In order to illustrate the technical solution of the present invention, a specific embodiment is used for illustration below.

[0050] Figure 1 This is a schematic diagram of the implementation process of a container resource restriction method provided in an embodiment of the present invention. The method is applied to a container orchestration platform. The container orchestration platform in this embodiment takes K8s as an example. Figure 1 , container resource restriction methods include:

[0051] S101, adding a setting configuration item for a data volume in a configuration file of a container group, wherein the setting configuration item is used for a container in the container group to restrict read and write resources of the data volume; and the data volume is mounted in a container of the container group.

[0052] Among them, the Pod configuration file can also be called the Pod definition file. The Pod configuration file is usually defined by a text-based markup language (YAML) file for data serialization. The file describes the relevant information of the Pod, including container images, ports, resource limits, volume mounts, etc.

[0053] In order to persist the data of the container, Kubernetes introduces the concept of Volume. The data volume can be mounted to one or more containers in the Pod, so that these containers can access and share the same data.

[0054] Common Volume types supported by Kubernetes include: EmptyDir (temporary storage volume), HostPath (host path volume), Network File System (NFS), PersistentVolume (PV), PersistentVolumeClaim (PVC), etc.

[0055] For example, you can use HostPathVolume to mount a directory on the host into a container. HostPath is to mount an actual directory on the Node host into the Pod for use by the container. This design ensures that the Pod is destroyed, but the data can still exist on the Node host.

[0056] The native K8s hostPath volume provides two configuration items:

[0057] path: Specifies the host path to be mounted.

[0058] type: specifies the mount type. The available types include Directory / File / BlockDevice, etc.

[0059] In the embodiment of the present application, a configuration item is added to the configuration file of the container group for the data volume, for example, as follows

[0060] As shown in Table 1, 4 setting configuration items are added:

[0061]

[0062]

[0063] Table 1

[0064] The configuration item is used to limit the read and write resources of the data volume for the containers in the container group. As shown in Table 1, the readBps configuration item is used to limit the read bandwidth of the container in the Pod to the device where the host path is located. For example, "10mb" means that the total read bandwidth of all containers in the Pod is limited to 10MB / second.

[0065] For example, the following is a partial example of a K8s Pod configuration file:

[0066]

[0067] In the above example, the Pod is named test, and the / root directory on the host is mounted to the / data directory of the container through the hostPath volume, so that the / root directory of the host can be accessed from the / data directory of the container. The four configuration items writeBps / readBps / writeIOps / readIOps added in this embodiment have the following functions:

[0068] Limit the maximum write bandwidth (writeBps) of the container to the / data directory to 10MBps;

[0069] Limit the container's maximum read bandwidth (readBps) for the / data directory to 10MBps;

[0070] Limit the container's maximum write IOPS (writeIOps) for the / data directory to 1000;

[0071] Limit the container's maximum read IOPS (readIOps) for the / data directory to 1000.

[0072] This embodiment is a Pod-level resource limit, and all containers in the Pod share this limit.

[0073] S102, parsing and setting the configuration item based on the data volume plug-in; the parsing and configuration actions are not performed through the standard interface of the container orchestration platform.

[0074] Kubelet uses a plug-in mechanism to manage different types of volumes. The kubelet volume plugin is a plug-in in Kubernetes that is used to manage data volumes in Pods. It allows Kubernetes administrators to mount different types of storage volumes (such as local storage, network storage, cloud storage, etc.) into Pods so that applications can access these volumes. The working principle of the kubelet volume plugin is that when the Kubernetes scheduler schedules a Pod to a node, the kubelet is responsible for starting the container in the Pod and mounting the data volume into the container.

[0075] The embodiment of the present application parses and sets the configuration items through the data volume plug-in, and the parsing and configuration actions are not performed through the standard interface of the container orchestration platform. Here, the standard interfaces provided by K8s include but are not limited to CNI, CSI, CRI, etc., and the embodiment of the present application does not impose any restrictions on the applicable standard interface types.

[0076] For example, when the embodiment of the present application implements parsing and setting through a data volume plug-in, two processes of configuration item parsing and setting can be added to the SetUp function, and then the parsing and setting processes in the SetUp function are executed through the data volume plug-in to implement IO resource restrictions on containers in the Pod. It should be understood that in K8s, these processes and plug-ins are executed by kubelet.

[0077] The SetUp function is used to perform some preparation work before the container is started. Specifically, it is usually used to perform the following tasks:

[0078] 1. Verify whether the configuration parameters of the data volume are correct and perform necessary parameter analysis.

[0079] 2. Check whether the data volume already exists. If not, create the data volume.

[0080] 3. Verify that the permissions and access control settings of the data volume are correct, and perform necessary permission checks.

[0081] 4. Prepare the mount point for the data volume and ensure the availability of the mount point.

[0082] The SetUp function prepares and configures data volumes in the kubelet volume plugin to ensure that the required data volumes can be correctly mounted before the container starts.

[0083] Compared with the related art that implements IO resource restriction by modifying the CRI interface definition, the CRI interface definition needs to be adapted and modified for different versions of K8s. The embodiment of the present application does not need to modify the CRI, but only needs to add a parsing and setting process in the SetUp function, and execute the parsing and setting process in the SetUp function through the data volume plug-in to achieve IO resource restriction for the container in the Pod. Compared with modifying the CRI interface definition, the embodiment of the present application only needs to modify the SetUp function, the solution is simple to change, and can be quickly ported to different K8s versions.

[0084] The embodiment of the present application adds a setting configuration item for the data volume in the configuration file of the container group. The setting configuration item is used for the containers in the container group to restrict the read and write resources of the data volume, and the data volume is mounted in the container of the container group. The setting configuration item is parsed and set based on the data volume plug-in, and the parsing and configuration actions are not performed through the standard interface of the container orchestration platform. The embodiment of the present application adds a setting configuration item to the configuration file of the container group, and parses and sets the setting configuration item through the data volume plug-in, so as to implement the reading and writing resource restriction of the data volume by the containers in the container group. It does not rely on the standard interface of the container orchestration platform, does not need to modify the interface definition, only needs to make simple changes, and can be quickly ported to different versions of the container orchestration platform, with wide versatility and compatibility.

[0085] In the related art, CRI implements resource restrictions, which are ultimately implemented through cgroup (control group). If implemented through CRI, the relevant interface definitions of CRI need to be modified, and different CRI implementations need to be adapted and adjusted. However, the embodiment of the present application uses a data volume plug-in to implement resource restrictions, such as Figure 2 As shown in the figure, when Pod uses HostPathVolume, you can directly use blkio cgroup in kubelet to perform QoS, skipping CRI, and no modification is required to CRI. Among them, kubelet uses the kube-apiserver component as a proxy to access services running on the cluster.

[0086] In one embodiment, the parsing and setting the configuration item based on the data volume plug-in includes:

[0087] Parsing the setting configuration item to obtain quality of service (QoS) configuration information in the setting configuration item;

[0088] Generate resource configuration information of the control group of the container group based on the QoS configuration information;

[0089] Based on the resource configuration information, the control group is updated.

[0090] The data volume plug-in parses the configuration item to obtain the QoS configuration information configured in the configuration item. In this embodiment, QoS refers to the quality of service control mechanism provided for the data volume. QoS is used to manage and control resource allocation and performance guarantee between different volumes in the storage system to ensure the availability and performance of key data. The QoS of the data volume usually includes the following aspects:

[0091] Bandwidth control: QoS can limit the bandwidth usage of data volumes to ensure that the bandwidth allocation of each data volume in the storage system is fair and reasonable. This can prevent a data volume from occupying too much bandwidth and affecting the performance of other data volumes.

[0092] Maximum number of disk reads and writes per second (IOPS) control: QoS can limit the IOPS usage of data volumes to ensure that the IOPS allocation of each data volume in the storage system is reasonable. This can prevent a data volume from using too much IOPS resources of the storage system, causing performance degradation of other data volumes.

[0093] As shown in Table 1, the QoS configuration information in the readBps configuration item is used to limit the read bandwidth of the container in the Pod to the device where the host path is located. For example, "10mb" means that the total read bandwidth of all containers in the Pod is limited to 10MB / second. If the value in the configuration item is empty, it means that it is not restricted.

[0094] This embodiment implements resource limitation through control groups (Cgroups). Cgroup is a function in the Linux kernel that is used to limit, record, and isolate resource usage (CPU, memory, disk I / O, network, etc.) of a group of processes.

[0095] After obtaining the QoS configuration information, the corresponding cgroup resource configuration resourceConfig (resource configuration information) is generated.

[0096] For example, for the QoS configuration information of the readBps configuration item in Table 1, the corresponding cgroup resourceConfig is generated as follows:

[0097] "BlkioThrottleReadBpsDevice":

[0098] {

[0099] Major: 0

[0100] Minor: 128

[0101] Rate:10485760

[0102] }

[0103] Finally, the path information of each subsystem in the control group is found, and the control group is updated to configure QoS according to the generated resource configuration information of the control group.

[0104] In one embodiment, the parsing the setting configuration item to obtain the QoS configuration information in the setting configuration item includes:

[0105] Determine whether QoS configuration information is configured in the setting configuration item;

[0106] If QoS configuration information is configured in the setting configuration item, then checking whether the volume type of the data volume is a valid type;

[0107] If the volume type of the data volume is a valid type, obtaining the major and minor device numbers of the block device where the data volume is located;

[0108] If the primary and secondary device numbers are normal, the configuration item is parsed successfully and the QoS configuration information is obtained.

[0109] For the specific process of parsing and setting configuration items of the data volume plug-in, please refer to Figure 3 , Figure 3 Schematic diagram of a configuration item parsing process provided by an embodiment of the present invention, including:

[0110] S301, parses the readBps configuration item and supports units such as B / KB / MB / GB.

[0111] S302, parse the writeBps configuration item and support units such as B / KB / MB / GB.

[0112] S303, parse the readlOps configuration item, which must be a decimal integer.

[0113] S304, parse the writelOps configuration item, which must be a decimal integer.

[0114] This embodiment takes the analysis of the four configuration items in Table 1 as an example, and parses the readBps, writeBps, readlOps, and writelOps configuration items respectively through the data volume plug-in.

[0115] S305, determine whether QoS is configured, that is, readBps, writeBps, readlOps, and writelOps are not all 0.

[0116] Among them, you can determine whether the QoS configuration information is configured by determining whether the value in the configuration item is 0. If it is not all 0, it means that the QoS configuration information is configured. For example, the value of the readBps configuration item is 10mb, which means that the QoS configuration information is configured. "10mb" means that the total read bandwidth of all containers in the Pod is limited to 10MB / second.

[0117] If QoS is configured, execute S306, otherwise execute S310.

[0118] S310, indicating that the QoS configuration item is not set.

[0119] If readBps, writeBps, readlOps, and writelOps are all 0, it means that no QoS configuration item is set.

[0120] S306, checking whether hostPath is a valid type, that is, whether Type is one of the five types: File / FileOrCreate / Directory / DirectoryOrCreate / BlockDevice.

[0121] When using a data volume of the hostPath type, you can set the Type field, which represents the volume type. By judging whether the value of the Type field is one of File / FileOrCreate / Directory / DirectoryOrCreate / BlockDevice, if it is, hostPath is considered to be a valid type. If not, it is not a valid type.

[0122] If it is a valid type, execute S307, otherwise execute S311.

[0123] S307, try to obtain the major and minor device numbers of the block device where the path is located.

[0124] Get the major device number and minor device number of the block device where the data volume is located. Block devices store information in fixed-size blocks, each of which has its own address. Typical block devices include disks. When a disk is registered to the operating system, it is assigned a major device number and a minor device number. The major device number represents the type of the device, and the minor device number represents the unique identifier of the device in the operating system.

[0125] S308, determining whether the acquisition fails or whether both major and minor are 0.

[0126] If the acquisition is successful, execute S309; ​​if the acquisition fails, or both major and minor are 0, execute S311.

[0127] S311: The path is not on the block device and QoS cannot be set.

[0128] Failed to obtain the major device number major and minor device number minor, or the major device number major and minor device number minor are both 0, indicating that the path is not on the block device, QoS setting is not supported, and the parsing process ends.

[0129] S309, the configuration item is parsed successfully and the QoS configuration is returned, including the major and minor device numbers, major and minor, and readBps, writeBps, readlOps, and writelOps.

[0130] If the major device number major and the minor device number minor are not both 0, it means that the major and minor device numbers are obtained successfully, the configuration items are parsed successfully, and the QoS configuration information is returned.

[0131] For example, the obtained QoS configuration information is as follows:

[0132] writeBps: "10mb", which means limiting the maximum write bandwidth (writeBps) of the container to the / data directory to 10MBps;

[0133] readBps: "10mb", which means limiting the maximum read bandwidth (readBps) of the container to the / data directory to 10MBps;

[0134] writeIOps: "1000" means that the maximum write IOPS (writeIOps) of the container to the / data directory is limited to 1000;

[0135] readIOps: "1000" means that the maximum read IOPS (readIOps) of the container for the / data directory is limited to 1000.

[0136] In one embodiment, updating the control group based on the resource configuration information includes:

[0137] Obtain a unique identifier of a control group of the container group;

[0138] Determining path information of each subsystem of the control group based on a unique identifier of the control group;

[0139] Based on the path information and the resource configuration information, parameters of each subsystem of the control group are set.

[0140] The specific process of implementing configuration item settings for the data volume plug-in can be referred to Figure 4 , Figure 4 This is a schematic diagram of a QoS setting process provided by an embodiment of the present invention, including:

[0141] S401: Whether QoS configuration items are set.

[0142] This embodiment connects Figure 3 In the illustrated embodiment, after the configuration item is parsed successfully, QoS configuration information is obtained, and if the QoS configuration information is obtained, S402 is continued to be executed.

[0143] Otherwise, execute S407. In S407, there is no need to configure QoS and the process can return directly.

[0144] S402, generate a corresponding cgroup resource configuration resourceConfig.

[0145] Generate the corresponding cgroup resource configuration information based on the QoS configuration information.

[0146] For example, according to the four configuration items readBps, writeBps, readlOps, and writelOps in Table 1, the resourceConfig of the generated cgroup is:

[0147] "BlkioThrottleReadBpsDevice":

[0148] {

[0149] Major: 0

[0150] Minor: 128

[0151] Rate:10485760

[0152] }

[0153] "BlkioThrottleWriteBpsDevice":

[0154] {

[0155] Major: 0

[0156] Minor: 128

[0157] Rate:10485760

[0158] }

[0159] "BlkioThrottleReadIOPSBpsDevice":

[0160] {

[0161] Major: 0

[0162] Minor: 128

[0163] Rate: 10000

[0164] }

[0165] "BlkioThrottleWriteIOPSBpsDevice":

[0166] {

[0167] Major: 0

[0168] Minor: 128

[0169] Rate: 10000

[0170] }

[0171] S403, obtain the cgroupManager used by kubelet.

[0172] Among them, cgroupManager is an encapsulation of cgroup-related operations and can support cgroup v1 and v2.

[0173] cgroupManager encapsulates cgroup operations, including but not limited to resource limits, priority settings, task management, and resource monitoring. Through cgroupManager, Kubernetes can achieve fine control over container resources to ensure efficient operation of containerized applications and reasonable resource allocation.

[0174] S404, obtain the cgroupName of the pod.

[0175] cgroupName is the unique identifier of the cgroup of the pod. The cgroupName of each pod is different. The cgroupName is bound to the absolute path of each cgroup subsystem of the pod. The absolute path of each cgroup subsystem of the pod can be calculated based on the cgroupName.

[0176] S405, using cgroupManager to update cgroup to configure Qos.

[0177] For example, update the cgroup with the following code:

[0178] cgroupManager.Update(&cm.CgroupConfig{Name:cgroupName,ResourceParameters:resourceConfig,}).

[0179] S406, QoS configuration is successful.

[0180] The cgroup manager is used to update the cgroup successfully, indicating that the QoS configuration is successful and the hostPath volume plugin completes the QoS setting.

[0181] In one embodiment, the parsing and setting the configuration item based on the data volume plug-in includes:

[0182] Determining the type of the data volume;

[0183] The setting configuration item is parsed and set based on the data volume plug-in corresponding to the type of the data volume.

[0184] Here, the type of data volume can be specified in the Pod's configuration file, and the type of data volume can be determined from the Pod's configuration file.

[0185] Common data volume types supported by Kubernetes include: EmptyDir (temporary storage volume), HostPath (host path volume), Network File System (NFS), PersistentVolume (PV), PersistentVolumeClaim (PVC), etc. When the volume type and configuration information are defined in the Pod, the kubelet will create and mount the data volume before creating the Pod container, and then provide it to the Pod container. The kubelet uses a plug-in mechanism to manage different types of volumes. For example, when a pod defines a hostPath volume, the kubelet will use the hostPath volume plug-in to manage the data volume.

[0186] According to the volume type and configuration information defined in the Pod, kubelet selects the appropriate volume plugin to mount the volume, and parses and sets the configuration items. For example, if a local storage volume is defined in the Pod, kubelet uses the hostPath volume plugin to mount the volume, and parses and sets the configuration items for the volume.

[0187] Kubernetes provides many built-in data volume plug-ins, such as emptyDir, hostPath, configMap, secret, etc., and also supports the extension of third-party data volume plug-ins. Administrators can select appropriate data volume plug-ins as needed and specify the type and configuration information of the data volume in the Pod configuration file.

[0188] In one embodiment, after parsing and setting the configuration item based on the data volume plug-in, the method further includes:

[0189] Adding process identifiers of containers in the container group to a subsystem of a control group of the container group.

[0190] After the data volume plug-in completes the parsing and setting of the configuration items, QoS has not yet taken effect because the Pod container has not yet been created. After the Pod container is created, the process identifiers (pid) of all containers in the Pod need to be added to the subsystem of the Pod's control group.

[0191] For example, you can add all container processes of a Pod to the blkio cgroup of the Pod in the SyncPod function.

[0192] The embodiment of the present application is a Pod-level resource restriction, which adds the process identifiers (pid) of all containers in the Pod to the Pod's blkio cgroup. The blkio cgroup is a control group subsystem that can limit the IOPS and throughput of processes (container processes in the Pod) reading and writing data volumes. The blkio cgroup allows system administrators to limit the I / O operations of different processes or process groups to ensure fair allocation of system resources and performance optimization.

[0193] The embodiment of the present application implements restrictions on the input / output operations of all containers in the Pod on the data volume by adding the process identifiers of all containers in the target container group to the Pod's blkiocgroup. All containers in the Pod share this restriction. For example, assuming that a Pod has 3 containers and the writeBps of hostPathvolume is set to 10m, then when these 3 containers use the hostPath volume at the same time, the total write bandwidth cannot exceed 10MBps.

[0194] For example, a readBps value of "10mb" means that the Pod's read bandwidth to the device where the host path is located is limited to 10MB / s, and the total read bandwidth of all containers in the Pod cannot exceed 10MBps when using the hostPath volume at the same time.

[0195] In one embodiment, adding the process identifier of the container in the container group to the subsystem of the control group of the container group includes:

[0196] Obtain a unique identifier of a control group of the container group;

[0197] Determining path information of each subsystem of the control group based on a unique identifier of the control group;

[0198] Based on the path information, process identifiers of containers in the container group are added to corresponding subsystems.

[0199] For example, you can add a QoS validation process to the SyncPod function, and use Kubelet to execute the QoS validation process in the SyncPod function to make QoS effective.

[0200] For specific procedures, please refer to Figure 5 , its QoS validation process includes:

[0201] S501, Kubelet executes the SyncPod function and completes the creation of all containers.

[0202] Among them, Kubelet executes the SyncPod function to complete the creation of the container in the Pod. The SyncPod function in kubelet is used to synchronize the Pod status. Specifically, it will update the Pod status information according to the Pod definition and the container status on the current node, including the container status, the number of container restarts, the container exit code, etc. At the same time, the SyncPod function will also start, stop, or restart the container according to the Pod definition to ensure that the Pod status is consistent with the definition.

[0203] S502, obtain cgroupManage used by kubelet.

[0204] Among them, cgroupManager is an encapsulation of cgroup related operations and can support cgroup v1 and v2.

[0205] cgroupManager encapsulates cgroup operations, including but not limited to resource limits, priority settings, task management, and resource monitoring. Through cgroupManager, Kubernetes can achieve fine control over container resources to ensure efficient operation of containerized applications and reasonable resource allocation.

[0206] S503, obtain the cgroupName of the pod.

[0207] Among them, cgroupName is the unique identifier of the cgroup of the pod. The cgroupName of each pod is different. The cgroupName is bound to the absolute path of each cgroup subsystem of the pod. The absolute path of each cgroup subsystem of the pod can be calculated based on the cgroupName.

[0208] S504, obtain the process IDs pids of all containers of the pod.

[0209] You can get the pod's container process pids through cgroupManager:

[0210] pids=cgroupManager.Pids(cgroupName).

[0211] S505, add all pids to the blkio cgroup of the pod.

[0212] First, the path information of each subsystem of the cgroup is determined through the cgroupName of the pod, and then the process identifier pids of the container is added to the corresponding subsystem based on the path information of the subsystem.

[0213] Equivalent to executing echo <pids>> / sys / fs / cgroup / blkio / <cgroupname> / cgroup.procs.

[0214] S506: QoS takes effect successfully.

[0215] Kubelet executes the SyncPod function and completes the QoS validation process. QoS is successfully validated. From then on, the reading and writing of data volumes by containers in the Pod are restricted by the QoS configuration information.

[0216] The embodiment of the present application is a QoS configuration performed at the Pod level. A Pod may have multiple containers, and these containers will share the same QoS. If you want to perform QoS at the container level, you can add a QoS configuration item in volumeMounts, so that for the same hostPath volume, multiple containers under the same Pod can be configured with different QoS, and then you need to perform QoS configuration on each container of the Pod separately and make it effective.

[0217] The embodiment of the present application solves the problem that K8s does not support HostPath Volume QoS. Only a small amount of modification is required to the K8s source code, and it can be easily ported to various K8s versions. At the same time, it does not rely on CRI, does not require any modification to CRI, supports Cgroup v1 and v2, and has wide versatility and compatibility.

[0218] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0219] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0220] It should be noted that the technical solutions described in the embodiments of the present invention can be arbitrarily combined without conflict.

[0221] In addition, in the embodiments of the present invention, “first”, “second”, etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0222] A container resource restriction device provided by an embodiment of the present invention includes:

[0223] An adding module, used for adding a setting configuration item for a data volume in a configuration file of a container group, wherein the setting configuration item is used for a container in the container group to restrict the read and write resources of the data volume; the data volume is mounted in a container of the container group;

[0224] A parsing and setting module is used to parse and set the configuration items based on the data volume plug-in; the parsing and configuration actions are not performed through the standard interface of the container orchestration platform.

[0225] In one embodiment, the parsing and setting module is specifically used for:

[0226] Parsing the setting configuration item to obtain quality of service QoS configuration information in the setting configuration item;

[0227] Generate resource configuration information of the control group of the container group based on the QoS configuration information;

[0228] Based on the resource configuration information, the control group is updated.

[0229] In one embodiment, the parsing and setting module is specifically used for:

[0230] Determine whether QoS configuration information is configured in the setting configuration item;

[0231] If QoS configuration information is configured in the setting configuration item, then checking whether the volume type of the data volume is a valid type;

[0232] If the volume type of the data volume is a valid type, obtaining the major and minor device numbers of the block device where the data volume is located;

[0233] If the primary and secondary device numbers are normal, the configuration item is parsed successfully and the QoS configuration information is obtained.

[0234] In one embodiment, the parsing and setting module is specifically used for:

[0235] Obtain a unique identifier of a control group of the container group;

[0236] Determining path information of each subsystem of the control group based on a unique identifier of the control group;

[0237] Based on the path information and the resource configuration information, parameters of each subsystem of the control group are set.

[0238] In one embodiment, the parsing and setting module is specifically used for:

[0239] Determining the type of the data volume;

[0240] The setting configuration item is parsed and set based on the data volume plug-in corresponding to the type of the data volume.

[0241] In one embodiment, the device further comprises:

[0242] An update module is used to add the process identifier of the container in the container group to the subsystem of the control group of the container group.

[0243] In one embodiment, the update module is specifically used to:

[0244] Obtain a unique identifier of a control group of the container group;

[0245] Determining path information of each subsystem of the control group based on a unique identifier of the control group;

[0246] Based on the path information, process identifiers of containers in the container group are added to corresponding subsystems.

[0247] In actual application, the parsing and setting module and the adding module can be implemented by a processor in an electronic device, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA).

[0248] It should be noted that: when the device provided in the above embodiment performs container resource restriction, only the division of the above modules is used as an example. In actual application, the above processing can be assigned to different modules as needed, that is, the internal structure of the device is divided into different modules to complete all or part of the above-described processing. In addition, the container resource restriction device provided in the above embodiment and the container resource restriction method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0249] The container resource restriction device can be in the form of an image file. After the image file is executed, it can be run in the form of a container or a virtual machine to implement the container resource restriction method described in this application. Of course, it is not limited to the image file form. As long as some software forms that can implement the container resource restriction method described in this application are within the protection scope of this application.

[0250] Based on the hardware implementation of the above program modules and in order to implement the method of the embodiment of the present application, the embodiment of the present application further provides an electronic device, and the above method is implemented by a processor of the electronic device. Figure 6 Schematic diagram of the hardware structure of the electronic device of the present application embodiment. Figure 6 As shown, the electronic equipment includes:

[0251] Communication interface, which can exchange information with other devices such as network equipment;

[0252] The processor is connected to the communication interface to realize information exchange with other devices, and is used to execute the method provided by one or more technical solutions of the electronic device side when running the computer program. The computer program is stored in the memory.

[0253] Of course, in actual applications, the various components in the electronic device are coupled together through the bus system. It is understandable that the bus system is used to achieve connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 In the specification, various buses are labeled as bus systems.

[0254] In addition, the electronic device of the present application may be in the form of a cluster, such as a cloud computing platform composed of clusters. The so-called cloud computing platform is a business form that uses virtualization technology to pool the resources of multiple terminals and then provides the required virtual resources and services.

[0255] The memory in the embodiment of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0256] It can be understood that the memory can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct memory bus random access memory (DRRAM). The memory described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memory.

[0257] The method disclosed in the above embodiment of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the program in the memory and completes the steps of the above method in combination with its hardware.

[0258] Optionally, when the processor executes the program, the corresponding processes implemented by the electronic device in each method of the embodiments of the present application are implemented, which will not be described in detail here for the sake of brevity.

[0259] In an exemplary embodiment, the embodiment of the present application further provides a computer program product, including a computer program, which can be executed by a processor of an electronic device to complete the steps described in the container resource limitation method in the embodiment of the present application.

[0260] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory storing a computer program, and the computer program can be executed by a processor of an electronic device to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0261] In the several embodiments provided in the present application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0262] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0263] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0264] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.

[0265] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0266] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0267] In addition, in the examples of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0268] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.< / cgroupname> < / pids>

Claims

1. A container resource limitation method, characterized in that: Applied to a container orchestration platform, the method includes: Adding a configuration item for the data volume in a configuration file of the container group, wherein the configuration item is used for the container in the container group to restrict the read and write resources of the data volume; the data volume is mounted in the container of the container group; The configuration items are parsed and set based on the data volume plug-in; and the parsing and configuration actions are not performed through a standard interface of the container orchestration platform.

2. The method according to claim 1, characterized in that The parsing and setting of the configuration items based on the data volume plug-in includes: Parsing the setting configuration item to obtain quality of service QoS configuration information in the setting configuration item; Generate resource configuration information of the control group of the container group based on the QoS configuration information; Based on the resource configuration information, the control group is updated.

3. The method according to claim 2, characterized in that The step of parsing the setting configuration item to obtain the QoS configuration information in the setting configuration item includes: Determine whether QoS configuration information is configured in the setting configuration item; If QoS configuration information is configured in the setting configuration item, then checking whether the volume type of the data volume is a valid type; If the volume type of the data volume is a valid type, obtaining the major and minor device numbers of the block device where the data volume is located; If the primary and secondary device numbers are normal, the configuration item is parsed successfully and the QoS configuration information is obtained.

4. The method according to claim 2, characterized in that: The updating of the control group based on the resource configuration information includes: Obtain a unique identifier of a control group of the container group; Determining path information of each subsystem of the control group based on a unique identifier of the control group; Based on the path information and the resource configuration information, parameters of each subsystem of the control group are set.

5. The method according to claim 1, characterized in that The parsing and setting of the configuration items based on the data volume plug-in includes: Determining the type of the data volume; The setting configuration item is parsed and set based on the data volume plug-in corresponding to the type of the data volume.

6. The method according to claim 1, characterized in that After parsing and setting the configuration item based on the data volume plug-in, the method further includes: Adding process identifiers of containers in the container group to a subsystem of a control group of the container group.

7. The method according to claim 6, characterized in that The adding the process identifier of the container in the container group to the subsystem of the control group of the container group includes: Obtain a unique identifier of a control group of the container group; Determining path information of each subsystem of the control group based on a unique identifier of the control group; Based on the path information, process identifiers of containers in the container group are added to corresponding subsystems.

8. A container resource restriction device, characterized in that: include: An adding module, used to add a setting configuration item for a data volume in a configuration file of a container group, wherein the setting configuration item is used for a container in the container group to restrict read and write resources of the data volume; The data volume is mounted in the container of the container group; A parsing and setting module is used to parse and set the configuration items based on the data volume plug-in; the parsing and configuration actions are not performed through the standard interface of the container orchestration platform.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the container resource limitation method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the container resource limitation method according to any one of claims 1 to 7.

Images