Rootfs quota management system, method, computer device and storage medium

Abstract

This specification provides a Rootfs quota management system, method, computer device, and storage medium, relating to the field of computer technology. The Rootfs quota management system includes a master control component, worker components, and instruction execution components corresponding to the worker components. The worker components can connect to the container runtime of their respective worker nodes. The master control component is used to obtain the quota request of a target task, which corresponds to a target container, and the target container mounts a corresponding root file system (Rootfs). Based on the quota request, it queries quota management data, determines the quota instruction, and sends the quota instruction to the target worker component. The worker component, when successfully connected to the container runtime, receives the quota instruction sent by the master control component. If the target container corresponding to the container identifier carried by the quota instruction exists and is running, it calls the instruction execution component to execute the quota at the writable layer of the target container's Rootfs. This improves the flexibility and compatibility of quota management.

Description

Technical Field

[0001] The embodiments described in this specification relate to the field of computer technology, specifically to a Rootfs quota management system, method, computer device, and storage medium. Background Technology

[0002] Container orchestration platforms are primarily used to automate the deployment, scaling, and operation of containerized applications, ensuring their stable operation in production environments. Kubernetes (K8s) is currently the mainstream container orchestration platform, capable of solving problems related to resource scheduling, fault recovery, and elastic scaling in container management.

[0003] In related technologies, when the Kubernetes platform limits the rootfs of containers, it often relies heavily on a specific container runtime. The container runtime is responsible for managing the entire lifecycle of the container, which is the foundation for running the container. It is responsible for pulling container images, starting and stopping containers, and providing the environment required for the container to run. Specific container runtimes can include Docker, Containerd Runtime, etc. This quota mechanism that relies on a specific container runtime can easily lead to related systems under the Kubernetes platform being bound to a specific runtime ecosystem, resulting in low flexibility and poor compatibility of storage resource quotas or limits.

[0004] Therefore, there is an urgent need to provide a rootfs quota management system to improve the flexibility and compatibility of storage resource quotas. Summary of the Invention

[0005] In view of this, various embodiments of this specification aim to provide a Rootfs quota management system, method, computer device, and storage medium to improve the flexibility and compatibility of storage resource quotas.

[0006] This specification provides a Rootfs quota management system, including a master control component, at least one working component, and an instruction execution component corresponding to each working component. Each working component can interact with both the master control component and its corresponding instruction execution component. The master control component is deployed on the master node of a container orchestration platform, and each working component and its corresponding instruction execution component are deployed on any working node of the container orchestration platform. Each working component can connect to the container runtime on its assigned working node. The master control component is used to obtain a quota request for a target task, wherein the target task corresponds to at least one target container, and the at least one target container is scheduled on one or more working nodes. The working component in the process is the target working component, and the target container is mounted with a corresponding root file system Rootfs. Based on the task quota request, the quota management data is queried to determine the quota instruction, wherein the quota management data describes or records quota information related to the task; the quota instruction is sent to the target working component; the working component is used to receive the quota instruction sent by the master component when the connection with the container runtime is successful, wherein the quota instruction carries a container identifier; if the target container corresponding to the container identifier carried by the quota instruction exists and is in a running state, the writable layer path of the root file system Rootfs mounted by the target container is obtained; the instruction execution component is called to execute the quota in the writable layer of the Rootfs of the target container.

[0007] In some implementations, the Rootfs quota management system further includes a task controller component and an APIServer component deployed on the master node. The APIServer component can interact with the master control component and the task controller component respectively. The task controller is used to receive the target task submitted by the user, use the APIServer component to create a target container corresponding to the target task, and schedule the target container to run on any of the worker nodes. When the target container is scheduled to run on any of the worker nodes, the task quota request is sent to the master control component.

[0008] In some implementations, the quota management data includes a quota policy configuration table and / or a task quota status table; the quota policy configuration table describes the correspondence between task types and default quotas, enabling differentiated quotas based on the task type of the target task; wherein, the task type is divided based on the storage space required by the task; the task quota status table is obtained by querying one or more of the following from the container orchestration platform: the running status, current quota, current usage, and node distribution of the container corresponding to the currently running task, using the APIServer component, so that the quota can be dynamically adjusted for the target task currently running.

[0009] This specification provides a Rootfs quota management method applied to the master control component of a Rootfs quota management system. The quota management system further includes at least one working component and an instruction execution component corresponding to each working component. The working components can interact with the master control component and the corresponding instruction execution component, respectively. The master control component is deployed on the master node of a container orchestration platform, and any working component and its corresponding instruction execution component are deployed on any working node of the container orchestration platform. The method includes: obtaining a task quota request for a target task; wherein the target task corresponds to at least one target container, the at least one target container is scheduled on one or more working nodes, the working component in the working node that schedules the target container is the target working component, and the target container has a corresponding root file system Rootfs mounted; querying quota management data based on the task quota request to determine a quota instruction; wherein the quota management data describes or records quota information related to the task; and sending the quota instruction to the target working component so that the target working component calls the instruction execution component to execute the quota in the writable layer of the Rootfs of the target container.

[0010] In some implementations, the task limit request is an initial limit request or a dynamic limit adjustment request, wherein the initial limit request carries the task type of the target task; the limit management data includes a limit policy configuration table, which describes the correspondence between task types and default limits; the step of querying the limit management data based on the task limit request to determine the limit instruction includes: when the task limit request is the initial limit request, querying the limit policy configuration table based on the task type of the target task to obtain a target limit value; and generating a limit instruction; wherein the limit instruction carries the target limit value.

[0011] In some implementations, the task limit request is an initial limit request or a dynamic limit adjustment request, wherein the dynamic limit adjustment request carries the task identifier of the target task and the requested limit value; the limit management data includes a task limit status table, which describes the current limit and current usage of the container corresponding to the currently running task; the step of querying the limit management data based on the task limit request to determine the limit instruction includes: when the task limit request is the dynamic limit adjustment request, querying the task limit status table based on the task identifier of the target task to obtain the current limit and current usage of the target container corresponding to the target task; if the requested limit value of the target task, the current limit and current usage of the target container meet the set limit adjustment conditions, using the requested limit value as the target limit value; generating a limit instruction; wherein the limit instruction carries the target limit value.

[0012] In some implementations, the task quota request carries a task identifier of the target task and a corresponding container identifier. The task identifier is used to uniquely identify the corresponding task, and the container identifier is used to uniquely identify the container of the corresponding task. The quota management data includes a task quota status table, which describes the node distribution of containers corresponding to currently running tasks or initialization tasks. The step of querying the quota management data based on the task quota request to determine the quota instruction includes: querying the task quota status table based on the task identifier and corresponding container identifier of the target task to determine the task status of the target task and the corresponding target working component; and generating the quota instruction if the task status of the target task is in a running state or an initialization state. The quota instruction carries the container identifier corresponding to the target task.

[0013] This specification provides a Rootfs quota management method applied to a working component of a Rootfs quota management system. The quota management system further includes a master control component and an instruction execution component corresponding to the working component. The working component can interact with both the master control component and the corresponding instruction execution component. The master control component is deployed on the master node of a container orchestration platform, and the working component and its corresponding instruction execution component are deployed on the working nodes of the container orchestration platform. The working component can connect to the container runtime on its working node. The method includes: upon successful connection to the container runtime, receiving a quota instruction sent by the master control component; wherein the quota instruction carries a container identifier; if the target container corresponding to the container identifier carried by the quota instruction exists and is running, obtaining the writable layer path of the root file system Rootfs mounted by the target container; and invoking the instruction execution component to execute the quota at the writable layer of the target container's Rootfs.

[0014] This specification provides a computer device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the Rootfs quota management method described in any of the above embodiments.

[0015] This specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the Rootfs quota management method described in any of the above embodiments.

[0016] In the various embodiments provided in this specification, firstly, the Rootfs quota management system includes a master control component, at least one working component, and an instruction execution component corresponding to each working component. The master control component is deployed on the master node of the container orchestration platform. Each working component and its corresponding instruction execution component are deployed on any working node of the container orchestration platform. Each working component can interact with both the master control component and its corresponding instruction execution component. Each working component can connect to the container runtime on its working node. Next, the master control component obtains a task quota request for a target task corresponding to at least one target container. This at least one target container is scheduled on one or more working nodes. The working component on the working node with the scheduled target container is called the target working component. Furthermore, each target container... The container mounts a corresponding root file system (Rootfs). Next, the master control component queries the quota management data based on the task quota request to determine the quota instruction and sends the quota instruction to the target worker component. Then, if the worker component successfully connects to the container runtime, it receives the quota instruction carrying the container identifier sent by the master control component. If the target container corresponding to the container identifier carried by the quota instruction exists and is running, the worker component obtains the writable layer path of the root file system (Rootfs) mounted by the target container and calls the instruction execution component to execute the quota at the writable layer of the target container's Rootfs. In this way, the flexibility and compatibility of storage resource quotas can be improved without relying on a specific type of container runtime when setting Rootfs quotas. Attached Figure Description

[0017] Figure 1 A schematic diagram of the Rootfs quota management system provided for the implementation of this specification;

[0018] Figure 2 An interactive diagram illustrating the Rootfs quota management method provided in the embodiments of this specification;

[0019] Figure 3 Another interactive diagram illustrating the Rootfs quota management method provided in the embodiments of this specification;

[0020] Figure 4 A flowchart illustrating the Rootfs quota management method provided in the embodiments of this specification;

[0021] Figure 5 A schematic diagram of a computer device provided for an embodiment of this specification. Detailed Implementation

[0022] To enable those skilled in the art to better understand the solutions described in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0023] In related technologies, taking the container orchestration platform Kubernetes (K8s) as an example, quota limits for containers rely on specific container runtimes such as Docker and Containerd. This means that the quota limit mechanism may fail when the container runtime needs to be changed. The flexibility and compatibility of quota limits on the K8s platform are relatively low, resulting in low storage resource utilization. Therefore, the flexibility and compatibility of container quota limits on container orchestration platforms need to be improved.

[0024] In some related technologies, a cluster can refer to an abstraction of a supercomputer, composed of nodes. A node can be a physical server or a virtual machine, and the Kubernetes (K8s) platform can be installed on each node. Kubernetes uses a master-slave architecture, meaning the nodes in the cluster can include a master node (represented by the Master), and other nodes are worker nodes (represented by Node or Worker). The master node is responsible for the scheduling, management, and operation of the cluster, while the worker nodes serve as the computing workload nodes within the cluster. In other words, Kubernetes includes a master node and one or more worker nodes corresponding to the master node.

[0025] This specification provides a Rootfs quota management system, which may include a master control component, at least one worker component, and an instruction execution component corresponding to each worker component. The following description uses the container orchestration platform Kubernetes as an example.

[0026] Please see Figure 1 , Figure 1This is a schematic diagram of the Rootfs quota management system provided in the embodiments of this specification. RootfsQuota-Master represents the master control component, RootfsQuota-Worker represents the worker component, and CMD-Agent represents the instruction execution component. The master control component can be deployed on the master node of the Kubernetes container orchestration platform. Any worker component and its corresponding instruction execution component can be deployed on any worker node of the Kubernetes container orchestration platform. Worker components can interact with both the master control component and their corresponding instruction execution components, and can connect to the container runtime on their respective worker nodes. For example, the container runtime can be Docker, Containerd, or other container runtimes, which are not limited here.

[0027] In this embodiment, the master control component can obtain the task limit request of the target task, determine the limit instruction based on the task limit request, and send the limit instruction to the target working component in at least one working component. If the working component successfully connects to the container runtime, it can receive the limit instruction sent by the master control component. If it determines that the corresponding target container exists and is running, it calls the instruction execution component to execute the limit on the target container's Rootfs.

[0028] The target task can be an artificial intelligence (AI) task or an AI-related task. For example, an AI task can be a phased task at different stages of the lifecycle, such as model development, model training, and model inference.

[0029] The target task corresponding to the task quota request obtained by the main control component has at least one target container. Alternatively, a running target task can correspond to one or more Pods, with a one-to-one correspondence between Pods and containers. This at least one target container is scheduled on one or more worker nodes, and the worker components within the worker nodes that schedule the target container are the target worker components. The target container has a corresponding root file system (Rootfs) mounted.

[0030] Specifically, the master control component can store or maintain quota management data, which describes or records quota information related to tasks. The master control component can query the quota management data based on task quota requests and determine the quota instruction based on the query results. Then, it sends the quota instruction to the target worker component. If the worker component successfully connects to the container runtime, it can receive the quota instruction carrying the container identifier sent by the master control component. If the target container corresponding to the container identifier carried by the quota instruction exists and is running, it obtains the writable layer path of the rootfs mounted by the target container and calls the instruction execution component to execute the quota at the writable layer of the target container's rootfs.

[0031] In the above implementation, by deploying a master control component on the master node and worker components and corresponding instruction execution components on the worker nodes, when setting quotas, firstly, the master control component obtains the task quota request for the target task corresponding to the target container. The target container of the target task is scheduled on one or more worker nodes. The worker component on the worker node with the scheduled target container is called the target worker component. Furthermore, each target container has a corresponding root file system (Rootfs). Next, the master control component queries quota management data based on the task quota request to determine the quota instruction and sends the quota instruction to the target worker component. Then, the worker component successfully connects to the container at runtime. In this case, the main control component sends a quota instruction carrying a container identifier. If the target container corresponding to the container identifier carried by the quota instruction exists and is running, the worker component obtains the writable layer path of the root file system Rootfs mounted by the target container and calls the instruction execution component to execute the quota in the writable layer of the target container's Rootfs. In this way, Rootfs quotas can be set without relying on a specific type of container runtime. This makes it compatible with multiple or any runtime environment such as CRI-O, Docker, Containerd, etc., reducing the problem of quota mechanism failure caused by runtime changes and improving the flexibility and compatibility of storage resource quotas.

[0032] In some implementations on the Kubernetes platform, worker components can run as DaemonSets, responsible for coordinating with instruction execution components to perform specific quota operations. Worker components can communicate with the master component via HTTP to receive quota instructions and receive feedback on their execution results. Worker components can also communicate with the corresponding container runtime via gRPC to obtain the status information of the container corresponding to the target task.

[0033] In some implementations, when a worker component calls an instruction execution component to execute a quota instruction, the instruction execution component executes it on its worker node (i.e., the host machine) in XFS format conforming to Rootfs requirements. This allows for operations such as querying, initializing, or adjusting quotas. The instruction execution component can return the execution result to the corresponding worker component. For example, the instruction execution component encapsulates the XFS instruction operations on its worker node or host machine, providing a unified interface via the SOCKS protocol for the corresponding worker components to call, ensuring the security and consistency of instruction execution. Simultaneously, it can parse and process the quota instruction execution result and return it to the worker component.

[0034] In some implementations, please refer to [the relevant documentation]. Figure 1 The Rootfs quota management system may also include a task controller component and an APIServer component deployed on the master node. The APIServer component can interact with the master control component and the task controller component, respectively.

[0035] Specifically, the task controller can receive user-submitted target tasks, create target containers corresponding to the target tasks using the APIServer component, and schedule the target containers to run on any worker node; in other words, it can create and schedule the Pods corresponding to the target tasks. After scheduling the target containers to run on any worker node, the task controller can send a task quota request to the master control component. This allows the master control component, worker components, and instruction execution components to collaboratively enforce quotas on the rootfs of the target containers for the target tasks. As an example, taking an AI task as the target task, the task controller component could be an AI task controller component. This enables flexible configuration of target tasks, or rather, flexible configuration of the target containers corresponding to the target tasks. By combining quota management data, the configuration flexibility and dynamic adjustment of quota policies for target tasks can be further improved, meeting diverse business needs, such as setting rootfs space quotas for target tasks running in Containerd, thus improving system adaptability.

[0036] In some implementations, quota management data may include a quota policy configuration table and / or a task quota status table.

[0037] In this embodiment, to reduce the situation of unreasonable resource allocation and inability to provide differentiated quota policies for different types of tasks, the quota policy configuration table can be used to describe the correspondence between task types and default quotas, or in other words, it can be used to describe the correspondence between task types and maximum quotas, so that when determining quota instructions based on quota management data and then setting quotas for the target container Rootfs, differentiated quotas based on the task type of the target task can be achieved.

[0038] For example, task types can be categorized based on the storage space required by the task. Continuing with the example of an AI task, the required storage space often varies at different stages of the AI ​​task's lifecycle, such as model development, model training, and model inference. Therefore, the task types in the quota policy configuration table can be determined based on the required storage space at different stages of the AI ​​task's lifecycle. That is, the task types in the quota policy configuration table can at least include model development, model training, and model inference, and each task type (model development, model training, and model inference) has a different default / maximum quota.

[0039] For example, the quota policy configuration table supports flexible and configurable quota policies. Different quota policies can be set according to the type of AI task to adapt to the needs of different AI tasks. Model development can provide developers with a relatively large quota, such as 20GB, to meet the needs of installing dependencies and storing temporary files during development. Model training can set a dynamic quota range based on the amount of training data and the model size, such as 10GB-50GB, which can be configured through parameters when submitting the AI ​​task. Model inference, which mainly involves reading data and outputting inference results, can be set with a smaller quota, such as 5GB. In this way, quotas can be allocated accordingly for different task types, optimizing resource allocation and significantly improving storage resource utilization.

[0040] In this implementation, the task quota status table can be obtained by querying worker nodes or using the APIServer component to query one or more of the following: the running status, current quota, current usage, and node distribution of the container corresponding to the currently running task in the Kubernetes container orchestration platform. This allows for dynamic adjustment of the quota for the currently running, already-limited target task based on the task quota status table when determining quota instructions based on quota management data and thus setting quotas for the target container's Rootfs. In this way, when the resource requirements of the program running in the target task's container change, users can adjust the quota according to the specific circumstances of different business scenarios and task requirements. This avoids the situation where the quota cannot be dynamically adjusted again during the container's entire lifecycle after the task's container starts, significantly improving task efficiency and operational stability.

[0041] For example, the master control component can be deployed on the Kubernetes cluster in the form of a Deployment. The master control component can communicate with external components via HTTP protocol / RESTful API, and interact with the cluster through a container orchestration platform such as the Kubernetes API client library to query relevant information about tasks and nodes.

[0042] This specification provides a Rootfs quota management method. The following description uses Kubernetes as the container orchestration platform and AI as the target task as an example. Please refer to [link / reference]. Figure 2 , Figure 2 This is an interactive diagram illustrating a Rootfs quota management method provided in this embodiment. This embodiment provides the operation steps of the method as shown in the interactive diagram, but based on conventional or non-creative labor, it may include more or fewer operation steps. The order of steps listed in the embodiment is merely one execution method among many, and does not represent the only execution order. In actual system or server products, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). This Rootfs quota management method can be applied to the main control component of a Rootfs quota management system, specifically as follows... Figure 2 As shown, the Rootfs quota management method may include the following steps.

[0043] Step S210: Obtain the task quota request for the target task; wherein, the target task corresponds to at least one target container, the at least one target container is scheduled on one or more worker nodes, the worker component in the worker node that has the target container scheduled is the target worker component, and the target container is mounted with the corresponding root file system Rootfs.

[0044] Specifically, the task controller can send a task limit request to the master control component via the HTTP protocol to request the limit configuration.

[0045] Step S220: Query the quota management data based on the task quota request to determine the quota instruction; wherein, the quota management data describes or records quota information related to the task.

[0046] Step S230: Send the quota instruction to the target worker component so that the target worker component calls the instruction execution component to execute the quota in the rootfs writable layer of the target container.

[0047] Specifically, the master component can send a quota limit instruction to the target worker component via the HTTP protocol, causing the worker component receiving the quota limit instruction to invoke the corresponding instruction execution component to execute the quota limit instruction. For example, the instruction execution component can execute in the writable layer of the target container's Rootfs in XFS form; in other words, the instruction execution component can execute the XFS quota limit instruction and return the execution result to the worker component. After receiving the execution result returned by the instruction execution component, the worker component can send the execution result back to the master component. After receiving the execution result from the worker component, the master component can return the quota limit result to the task controller, which then determines whether to continue with subsequent tasks.

[0048] For example, the task controller is configured with a corresponding platform interface, through which users can interact with the task controller, and the task controller can display the quota results through the platform interface. The quota result includes quota failure or quota success. If the quota fails, the task controller can notify the user and terminate the job.

[0049] In some implementations, a task limit request may carry at least one or more of the following: a task identifier, a container identifier, and a task type. The task identifier uniquely identifies the corresponding task, and the container identifier uniquely identifies the container containing the corresponding task. The task identifier can be a task ID, and the container identifier can be a container ID. The task limit request can be an initial limit request or a dynamically adjusted limit request.

[0050] Quota management data can include a quota policy configuration table and a task quota status table. The quota policy configuration table describes the correspondence between task types and default quotas, while the task quota status table describes the current quota, current usage, and node distribution of the container corresponding to the currently running task or initialization task.

[0051] In some implementations, the initialization limit request may at least carry the task type of the target task. Step S220, querying limit management data based on the task limit request to determine the limit instruction, may include the following steps S310-S320.

[0052] Step S310: If the task limit request is an initial limit request, query the limit policy configuration table based on the task type of the target task to obtain the target limit value.

[0053] In some cases, please refer to [the relevant documentation]. Figure 2 Users can submit target tasks to the task controller. These target tasks can be AI tasks and include information such as task type and task name. After receiving the target task, the task controller can use the APIServer component in Kubernetes to create the target container / Pod corresponding to the target task and schedule it to run on any worker node. Then, the task controller can send an initial quota request to the master control component via the HTTP protocol.

[0054] Specifically, the main control component can query the quota policy configuration table based on the task type of the target task carried in the initial quota request to obtain the default / maximum quota corresponding to the task type of the target task, and use it as the target quota value. For example, the request parameters for the initial quota request may include: "task-id": "a16514557146296320444804", "task-type": model-training. Here, task-id represents the task identifier, task-type represents the task type, and model-training indicates that the task type is model training.

[0055] Step S320: Generate a limit instruction; wherein the limit instruction carries a target limit value.

[0056] For example, after generating the limit, the master control component can send a limit instruction to the corresponding target worker node, instructing the target worker node to coordinate with the instruction execution component to limit the target container in the form of SFX. The parameters carried by the limit instruction may include: "container-id": "f9ae6ce6d1a2c393f969f539bddc719ef93d689736be002a3c612a844add9449", "size": "40G". Here, container-id represents the container identifier, and size represents the target limit value.

[0057] In some implementations, the dynamic adjustment limit request may at least carry the task identifier of the target task and the requested limit value. In step S220, querying the limit management data based on the task limit request to determine the limit instruction may include the following steps S410-S430.

[0058] Step S410: When the task limit request is a dynamically adjusted limit request, query the task limit status table based on the task identifier of the target task to obtain the current limit and current usage of the target container corresponding to the target task.

[0059] In some cases, please refer to Figure 3 Users can initiate dynamic limit adjustment requests to the task controller via the platform interface or API to request adjustments to the limits of a target task, such as requesting an increase or decrease in the current limit of the container corresponding to the target task. The dynamic limit adjustment request carries the target size of the requested limit value. After receiving the user's dynamic limit adjustment request, the task controller can forward it to the main control component.

[0060] Specifically, after receiving a request to dynamically adjust the limit, the main control component can obtain information such as the current limit, current usage, and remaining disk space of the target container corresponding to the target task by querying the task limit status table, the APIServer component, or the worker node.

[0061] Step S420: If the application limit value of the target task, the current limit of the target container, and the current usage meet the set limit adjustment conditions, the application limit value will be used as the target limit value.

[0062] Specifically, if the requested limit is less than the target task's current limit and greater than the current usage, the limit adjustment conditions are met, and the worker component can then coordinate with the instruction execution component to implement the limit. Otherwise, a failure is returned, informing the user of the reason, such as the requested limit being too small to meet actual usage. Here, the current limit refers to the target task's current Rootfs limit; the current usage refers to the target task's current actual Rootfs size.

[0063] If the requested limit is greater than the current limit of the target task and less than the remaining disk space, the limit adjustment conditions are met, and the work component can then coordinate with the instruction execution component to implement the limit. Otherwise, a failure is returned, and the user is prompted with the reason, such as insufficient disk space, which prevents the application from being fulfilled.

[0064] Step S430: Generate a limit instruction; wherein the limit instruction carries a target limit value.

[0065] For example, when the task limit request is a dynamically adjusted limit request, the resulting limit instruction is also an adjustment instruction.

[0066] In some implementations, the task quota request carries the task identifier of the target task and the corresponding container identifier. Step S220, querying quota management data based on the task quota request to determine the quota instruction, may include the following steps S510-520.

[0067] Step S510: Based on the task identifier and corresponding container identifier of the target task, query the task quota status table to determine the task status of the target task and the corresponding target working component.

[0068] In some cases, please refer to [the relevant documentation]. Figure 2 or Figure 3 After receiving a task limit request, such as an initial limit request or a dynamic limit adjustment request, the main control component can obtain the task status of the target task and the node information of the target working component corresponding to the target task by querying the task limit status table or the APIServer component.

[0069] Step S520: If the target task is in the running or initialization state, generate a quota instruction; wherein the quota instruction carries the container identifier corresponding to the target task.

[0070] If the target task is not running, the main control component can return an error message to the user, indicating that the target task is not running and cannot be subject to quota.

[0071] This specification provides a Rootfs quota management method, which can be applied to working components within a Rootfs quota management system. Please refer to... Figure 4 The Rootfs quota management method may include the following steps.

[0072] Step S610: If the connection with the container runtime is successful, receive the quota instruction sent by the master control component; wherein the quota instruction carries the container identifier.

[0073] Specifically, please refer to Figure 4 During the Rootfs quota management process, the worker nodes and corresponding worker components need to be started first. After the worker components are started, when connecting to the container via gRPC, if the connection fails, an exponential backoff reconnection can be performed until the connection is successful.

[0074] Exponential backoff reconnection can refer to reconnecting at a specified time interval (0, 1, 2, 3... n) after the current connection failure, until the maximum reconnection time is reached. For example, if the connection fails at the current time, the specified time interval is 5 seconds, and the maximum reconnection time interval is 5 minutes, then reconnection can occur 1 second, 5 seconds, 25 seconds... after the current time, until 5 minutes are reached or the connection is successfully established.

[0075] Step S620: If the target container corresponding to the container identifier carried by the quota instruction exists and is in a running state, obtain the writable layer path of the root file system Rootfs mounted by the target container.

[0076] Specifically, after the working component successfully connects with the container runtime, it can wait for the quota instruction. Upon receiving the quota instruction, if the target container does not exist or is not running, it returns failure; if it exists and is running, it obtains the writable layer path Upperdir of the target container.

[0077] Step S630: Invoke the instruction execution component to execute the quota in the writable layer of the rootfs of the target container.

[0078] Specifically, after the working component calls the instruction execution component to execute the quota, it can return the execution result to the master component.

[0079] In the above implementation, through the coordinated operation of the main control component, the working components, and the instruction execution component, the initialization or dynamic adjustment of the Rootfs quota for a single container can be achieved, such as to meet the scaling requirements midway through an AI training task. Furthermore, it enables direct calls to xfs at the host machine level via independent services, improving runtime independence.

[0080] In the above implementation, through the collaborative cooperation between the master control component, the working component, and the instruction execution component, the integration of quota logic into the native scheduling process of the container orchestration platform Kubernetes can be improved while avoiding deep modification or extension of the container runtime, which would increase its complexity and maintenance costs.

[0081] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a computer, implements the Rootfs quota management method in any of the above embodiments.

[0082] This specification also provides a computer program product containing instructions that, when executed by a computer, cause the computer to implement the Rootfs quota management method in any of the above embodiments.

[0083] This specification also provides a computer device, including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the Rootfs quota management method in any of the above embodiments.

[0084] In some implementations, please refer to Figure 5 The computer device can be a terminal, and its internal structure diagram can be as follows: Figure 5 As shown. The computer device includes a processor, memory, and communication interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a rootfs quota management method.

[0085] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments described herein, and are not intended to limit the scope of the invention.

[0086] It is understood that in the various embodiments described in this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments described in this specification.

[0087] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and the implementation methods in this specification are not limited in this respect.

[0088] Unless otherwise stated, all technical and scientific terms used in the embodiments of this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0089] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0090] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0093] The above description is merely a specific embodiment of this specification, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this specification should be included within the scope of protection of this specification. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A Rootfs quota management system, characterized in that, The system includes a master control component, at least one working component, and an instruction execution component corresponding to each working component. The working components can interact with the master control component and the corresponding instruction execution component, respectively. The master control component is deployed on the master node of the container orchestration platform, and each working component and its corresponding instruction execution component are deployed on any working node of the container orchestration platform. The working component can connect with the container runtime in the working node it is located on. The main control component is used to obtain the task quota request of the target task, wherein the target task corresponds to at least one target container, the at least one target container is scheduled on one or more of the worker nodes, the worker component in the worker node that has the target container scheduled is the target worker component, and the target container is mounted with a corresponding root file system Rootfs; based on the task quota request, query quota management data to determine quota instructions, wherein the quota management data describes or records quota information related to the task; and send the quota instructions to the target worker component. The working component is configured to receive a quota instruction sent by the master control component when the connection with the container runtime is successful, wherein the quota instruction carries a container identifier; if the target container corresponding to the container identifier carried by the quota instruction exists and is in a running state, obtain the writable layer path of the root file system Rootfs mounted by the target container; and call the instruction execution component to execute the quota in the writable layer of the rootfs of the target container.

2. The system according to claim 1, characterized in that, The Rootfs quota management system also includes a task controller component and an APIServer component deployed on the master node. The APIServer component can interact with the master control component and the task controller component respectively. The task controller is used to receive the target task submitted by the user, create a target container corresponding to the target task using the APIServer component, and schedule the target container to run on any of the worker nodes; when the target container is scheduled to run on any of the worker nodes, it initiates the task quota request to the main control component.

3. The system according to claim 2, characterized in that, The quota management data includes a quota policy configuration table and / or a task quota status table; The quota policy configuration table is used to describe the correspondence between task types and default quotas, so as to enable differentiated quotas based on the task type of the target task; wherein, the task type is divided based on the storage space required by the task. The task limit status table is obtained by querying one or more of the following from the container orchestration platform: the running status, current limit, current usage, and node distribution of the container corresponding to the currently running task, using the APIServer component. This enables dynamic adjustment of the limit for the target task that is currently running.

4. A Rootfs quota management method, characterized in that, A master control component is applied to a Rootfs quota management system. The quota management system further includes at least one working component and an instruction execution component corresponding to each working component. Each working component can interact with the master control component and its corresponding instruction execution component. The master control component is deployed on the master node of a container orchestration platform, and each working component and its corresponding instruction execution component are deployed on any working node of the container orchestration platform. The method includes: Obtain the task quota request for the target task; wherein, the target task corresponds to at least one target container, the at least one target container is scheduled on one or more of the worker nodes, the worker component in the worker node that has the target container scheduled is the target worker component, and the target container is mounted with a corresponding root file system Rootfs; Based on the task limit request, query the limit management data to determine the limit instruction; wherein, the limit management data describes or records limit information related to the task; The quota instruction is sent to the target working component so that the target working component invokes the instruction execution component to execute the quota in the rootfs writable layer of the target container.

5. The method according to claim 4, characterized in that, The task limit request is either an initial limit request or a dynamic limit adjustment request. The initial limit request carries the task type of the target task. The limit management data includes a limit policy configuration table, which describes the correspondence between task types and default limits. The step of querying the limit management data based on the task limit request to determine the limit instruction includes: If the task limit request is the initial limit request, the limit policy configuration table is queried based on the task type of the target task to obtain the target limit value; Generate a limit instruction; wherein the limit instruction carries the target limit value.

6. The method according to claim 4, characterized in that, The task limit request is either an initial limit request or a dynamic limit adjustment request. The dynamic limit adjustment request carries the task identifier and requested limit value of the target task. The limit management data includes a task limit status table, which describes the current limit and current usage of the container corresponding to the currently running task. The step of querying the limit management data based on the task limit request to determine the limit instruction includes: When the task limit request is the dynamically adjusted limit request, the task limit status table is queried based on the task identifier of the target task to obtain the current limit and current usage of the target container corresponding to the target task; If the application limit value of the target task, the current limit of the target container, and the current usage meet the set limit adjustment conditions, the application limit value will be used as the target limit value; Generate a limit instruction; wherein the limit instruction carries the target limit value.

7. The method according to claim 4, characterized in that, The task quota request carries the task identifier of the target task and the corresponding container identifier. The task identifier is used to uniquely identify the corresponding task, and the container identifier is used to uniquely identify the container of the corresponding task. The quota management data includes a task quota status table, which is used to describe the node distribution of the containers corresponding to currently running tasks or initialization tasks. The step of querying the limit management data based on the task limit request to determine the limit instruction includes: Based on the task identifier and the corresponding container identifier of the target task, the task limit status table is queried to determine the task status of the target task and the corresponding target working component; When the target task is in a running or initialization state, the quota instruction is generated; wherein the quota instruction carries the container identifier corresponding to the target task.

8. A rootfs quota management method, characterized in that, A working component is applied to a Rootfs quota management system. The quota management system further includes a master control component and a corresponding instruction execution component. The working component can interact with both the master control component and the corresponding instruction execution component. The master control component is deployed on the master node of a container orchestration platform. The working component and its corresponding instruction execution component are deployed on the working nodes of the container orchestration platform. The working component can connect to the container runtime within its working node. The method includes: Upon successful connection with the container runtime, a limit instruction sent by the master control component is received; wherein the limit instruction carries a container identifier; If the target container corresponding to the container identifier carried by the quota instruction exists and is in a running state, obtain the writable layer path of the root file system Rootfs mounted by the target container. The instruction execution component is invoked to execute the quota in the writable Rootfs layer of the target container.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the Rootfs quota management method according to any one of claims 4 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the Rootfs quota management method according to any one of claims 4 to 8.

Images