Persistent volume capacity adjustment method, k8s cluster management system, storage medium
Patent Information
- Application Number
- CN202611080117.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-21
- Publication Date
- 2026-08-18
AI Technical Summary
[0004]然而，该方案必须通过重启应用程序容器才能完成新旧持久卷的切换，会直接造成业务中断，无法满足高可用业务的连续性要求
Smart Images
Figure CN122601668A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a persistent volume capacity adjustment method, a K8s cluster management system, and a storage medium. Background Technology
[0002] With the rapid popularization of cloud-native architecture, Kubernetes (K8s) has become the mainstream platform in the field of container orchestration. Persistent volumes, as the key carriers for storing application data, logs and other core business data in Kubernetes, are being used on an ever-increasing scale.
[0003] Currently, the most mainstream technical solution in the industry for scaling down Kubernetes persistent volumes is to first create a new persistent volume declaration and corresponding persistent volume with a smaller capacity, then stop all application containers using the old persistent volume, completely copy the data in the old persistent volume to the new persistent volume, then modify the container configuration file to mount the new persistent volume, and finally restart the container and delete the old persistent volume declaration and persistent volume.
[0004] However, this solution requires restarting the application container to switch between the old and new persistent volumes, which directly causes business interruption and fails to meet the continuity requirements of high-availability services. Summary of the Invention
[0005] In view of the above-mentioned defects or deficiencies in the existing technology, it is desirable to provide a persistent volume capacity adjustment method, a K8s cluster management system, and a storage medium that can adjust the quota information of the dedicated storage directory corresponding to the target persistent volume to the target shrinkage capacity by determining the dedicated storage directory corresponding to the target persistent volume. This achieves the goal of not triggering the restart operation of the application container during the shrinkage process, ensuring that the read and write operations of the business are uninterrupted and imperceptible throughout the process.
[0006] Firstly, this application provides a persistent volume capacity adjustment method, applied to a K8s cluster management system, the method comprising: The management system determines the target managed capacity of the target persistent volume and the identification information of the persistent volume declaration corresponding to the target persistent volume; If the target management capacity is the target shrinkage capacity, then: The dedicated storage directory for the persistent volume declaration is determined based on the identification information. The dedicated storage directory is used to uniquely identify the actual storage location of the target persistent volume in the corresponding target storage service. By calling the target storage service's directory configuration interface through the first standard interface, the quota information associated with the dedicated storage directory is adjusted to the target shrinkage capacity. The first standard interface is a unified interface for the management system to configure multi-source heterogeneous storage services. If the target management capacity is the target expansion capacity, then the quota information of the persistent volume declaration is modified based on the target expansion capacity.
[0007] In conjunction with the first aspect, in one possible implementation, the management system determines the target managed capacity of the target persistent volume and the identification information of the persistent volume declaration corresponding to the target persistent volume, specifically as follows: The management system receives a user's first capacity adjustment request, which includes identification information and the target management capacity; or The current capacity utilization of the target persistent volume is periodically checked based on a preset time interval. A second capacity adjustment request is generated based on the current capacity utilization rate. The second capacity adjustment request includes identification information and target management capacity.
[0008] In conjunction with the first aspect, one possible implementation involves generating a second capacity adjustment request based on the current capacity utilization rate, specifically as follows: If the current capacity utilization rate of a consecutive preset number of units is less than or equal to the first threshold, then the target management capacity is set to the first preset value, and a first-level reduction request is generated based on the identification information and the first preset value. If the current capacity utilization rate of a consecutive preset number of units is greater than the first threshold and less than or equal to the second threshold, then the target management capacity is set to the second preset value, and a secondary reduction request is generated based on the identification information and the second preset value. If the current capacity utilization rate of a consecutive preset number of units is greater than or equal to the third threshold and less than the fourth threshold, then the target management capacity is set to the third preset value, and a first-level expansion request is generated based on the identification information and the third preset value. If the current capacity utilization rate of a consecutive preset number of units is greater than or equal to the fourth threshold, the target management capacity is set to the fourth preset value, and a secondary expansion request is generated based on the identification information and the fourth preset value, wherein the first threshold < the second threshold < the third threshold < the fourth threshold.
[0009] In conjunction with the first aspect, in one possible implementation, before determining the dedicated storage directory for the persistent volume claim based on the identification information, the method further includes: If the first capacity adjustment request / second capacity adjustment request is a scaling-down request, then determine whether the target management capacity in the first capacity adjustment request / second capacity adjustment request meets the preset rules. The preset rules are that the target management capacity is greater than or equal to the first real-time usage capacity and the target management capacity is greater than or equal to the preset minimum capacity. The minimum capacity is used to represent the minimum quota value of the target persistent volume in the target storage service. If not, then reject the first capacity adjustment request / second capacity adjustment request.
[0010] In conjunction with the first aspect, in one possible implementation, after determining the dedicated storage directory for the persistent volume declaration based on the identification information, the method further includes: If the first capacity adjustment request / second capacity adjustment request is a reduction request, the target managed capacity is verified again based on the second real-time usage capacity. If it fails, the reduction request is rejected.
[0011] In conjunction with the first aspect, one possible implementation method also includes: Update the annotation information in the persistent volume declaration based on the target expansion or reduction capacity. The annotation information includes the quota information of the target persistent volume before reduction or expansion.
[0012] In conjunction with the first aspect, one possible implementation method also includes: If the update fails, the disaster recovery rollback mechanism will be triggered to restore the capacity of the target persistent volume to its previous size before shrinking or expanding.
[0013] Secondly, this application provides a K8s cluster management system, the management system including: The dual capacity verification module is configured to determine the target managed capacity of the target persistent volume and the identification information of the persistent volume declaration corresponding to the target persistent volume; The lossless scaling-down execution module is configured to determine the dedicated storage directory of the persistent volume declaration based on the identification information when the target management capacity is the target scaling-down capacity. The dedicated storage directory is used to uniquely identify the actual storage location of the target persistent volume in the corresponding target storage service. The lossless scaling-down execution module is also configured to call the target storage service's directory configuration interface through the first standard interface to adjust the quota information associated with the dedicated storage directory to the target scaling-down capacity. The first standard interface is a unified interface configured by the management system for multi-source heterogeneous storage services. The Kubernetes management layer is configured to modify the quota information of persistent volume declarations based on the target expansion capacity when the target managed capacity is the target expansion capacity.
[0014] In conjunction with the second aspect, in one possible implementation, the management system also includes: The intelligent elastic scheduling module is configured to periodically detect the current capacity utilization of the target persistent volume based on a preset time interval; The intelligent elastic scheduling module is also configured to generate a second capacity adjustment request based on the current capacity utilization rate. The second capacity adjustment request includes identification information and target management capacity. The dual capacity verification module is configured to receive a user's first capacity adjustment request or a second capacity adjustment request, which includes identification information and target management capacity.
[0015] Thirdly, this application also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the persistent volume capacity adjustment method as described in any of the first aspects. Attached Figure Description
[0016] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an example of the architecture diagram of the K8s cluster management system provided in this application embodiment; Figure 2 This is a flowchart illustrating the persistent volume capacity adjustment method provided in one embodiment of the present application. Figure 3 This is a schematic diagram of another process of the persistent volume capacity adjustment method provided in this application embodiment, as one example. Figure 4 This is a schematic diagram of another process of the persistent volume capacity adjustment method provided in this application embodiment, as one example. Detailed Implementation
[0017] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.
[0018] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments. Furthermore, the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The terms "first" and "second," etc., in the specification and claims of the embodiments of this application are used to distinguish different objects, not to describe a specific order of objects.
[0019] The following explains the terms used in this application: Kubernetes: A container-based cluster management system built on top of container runtime environments such as Docker and containers. It is used to manage containerized workloads and services. It is a portable, scalable, open-source platform with a large and rapidly growing ecosystem, and its services, support, and tools are widely available.
[0020] Pod: In Kubernetes, the smallest deployable unit of computing is created and managed. It contains one or more containers that share storage, networking, and declarations of how to run them. A Pod node models an application-specific "logical host" containing one or more application containers that are relatively tightly coupled together.
[0021] A persistent volume (PV) is a block of storage within a cluster that can be provisioned in advance by the administrator or dynamically provisioned using a storage class. Persistent volumes are cluster resources, just as nodes are cluster resources. Like regular volumes, PVs are implemented using volume plugins, but they have a lifecycle independent of any Pod using the PV.
[0022] A Persistent Volume Claim (PVC) expresses a user's request for storage. Conceptually, it's similar to a Pod. A Pod consumes node resources, while a PVC claim consumes PV resources. A Pod can request a specific amount of resources (CPU and memory); similarly, a PVC claim can request a specific size and access mode.
[0023] StorageClass: Provides administrators with a way to describe storage "classes". Different types may map to different quality of service levels or backup policies, or any policy defined by the cluster administrator.
[0024] Annotations: Kubernetes metadata annotations allow you to attach metadata to Kubernetes objects using tags or annotations, enabling client programs (such as tools and libraries) to access this metadata information.
[0025] Lossless scaling down: Reducing the capacity of persistent volumes without restarting Pods, interrupting business operations, or migrating data.
[0026] With the rapid adoption of cloud-native architectures, Kubernetes has become the mainstream platform in the container orchestration field. Persistent volumes, as a key carrier for storing application data, logs, and other core business data in Kubernetes, are seeing their usage grow rapidly. The Kubernetes platform provides a general container storage interface, enabling dynamic configuration of persistent storage and automated processing of storage requests. Among these, the persistent volume declaration and persistent volume storage management mechanism has become the standard implementation method for data persistence in containerized applications.
[0027] Currently, the most mainstream technical solution for scaling down Kubernetes persistent volumes is to rebuild the persistent volume and persistent volume declaration and then migrate the data. The specific implementation steps are as follows: The first step is to create a new persistent volume declaration and the corresponding persistent volume with a capacity that meets the scaling-down requirements; The second step is to stop all Pods that are currently using the old persistent volume declaration to avoid data inconsistency issues during the data migration process. If the Pods are not manually paused, the system will automatically trigger the Pods to restart when the new persistent volume declaration is mounted later, but there is a risk of data loss. The third step is to completely migrate the business data stored in the old persistent volume to the newly created persistent volume; The fourth step is to modify the target Pod's configuration file to point its storage mount to the newly created persistent volume declaration; Fifth, restart the application Pod to load the new storage configuration and mount the new persistent volume; Step 6: After confirming that the business is running normally and the data is complete and error-free, delete the old persistent volume declaration and persistent volume to release the underlying storage resources.
[0028] The aforementioned existing technical solutions have the following obvious drawbacks: First, this solution requires restarting the application Pod to complete the switch between the old and new persistent volumes, which will directly cause business interruption and cannot meet the continuity requirements of high-availability services. Secondly, the data migration process lacks necessary security verification mechanisms, resulting in extremely high migration risks and making it very easy to cause data loss or data inconsistency. Finally, this solution relies entirely on manual operation, lacks the ability to automatically scale down persistent volumes, has extremely high operation and maintenance costs, and cannot meet the batch storage scaling down operation and maintenance needs of large-scale Kubernetes clusters.
[0029] Based on this, embodiments of this application provide a persistent volume capacity adjustment method, a K8s cluster management system, and a storage medium, which can adjust the quota information of the dedicated storage directory to the target shrinkage capacity by determining the dedicated storage directory of the persistent volume, so as to achieve the effect of not triggering the restart operation of the application container during the shrinkage process, and ensuring that the read and write operations of the business are uninterrupted and imperceptible throughout the process.
[0030] The method provided in this application embodiment can be applied to, for example... Figure 1 The K8s cluster management system 1 shown has a network architecture divided into four layers, from top to bottom: Kubernetes management layer 10, capacity control layer 20, storage adaptation layer 30, and storage layer 40. Each layer is decoupled and interacts only through standardized interfaces to ensure the lightweight, universality, and scalability of the network architecture.
[0031] Kubernetes Management Layer 10: This is the business layer of the Kubernetes cluster, which includes Pods and PVCs / PVs. The method in this application does not modify any core configuration of this layer during execution, such as not restarting Pods or rebuilding PVs / PVCs, and maintains the original mounting relationships and business operation status. Specifically, Kubernetes management layer 10 is a built-in functional unit that is configured to receive capacity adjustment requests initiated by external users and the system when the capacity adjustment request is an expansion request, and parse out the identifier information of the persistent volume declaration and the target managed capacity.
[0032] In this application, the Kubernetes management layer 10 is responsible for the overall management and control of the entire lifecycle of core resources such as PVC, PV, and Pod within the cluster. After the Kubernetes management layer 10 parses and obtains the identification information and target management capacity of the persistent volume declaration, it distributes them to the unified scaling management module 201.
[0033] Unified Management Module for Expansion and Reduction 201: is configured to send identification information and target management capacity to Synchronization and Disaster Recovery Rollback Module 205; The synchronization and disaster recovery rollback module 205 is configured to update the metadata annotations in the persistent volume declaration based on the identification information and the target managed capacity. The metadata annotations include the quota information of the target persistent volume before expansion. The Kubernetes management layer 10 is also configured to follow the native Kubernetes process, modify the spec.resources capacity configuration of the PVC based on the identification information in the scaling command, and complete the scaling based on the native Kubernetes mechanism.
[0034] Capacity Control Layer 20: The core functional layer of this invention, configured to receive a capacity adjustment request automatically generated by the user or system when the target managed capacity is the target shrinkage capacity, and then execute specific capacity adjustment logic. The capacity adjustment logic includes at least one of request verification, execution of target persistent volume capacity adjustment, and synchronization of persistent volume declaration annotation information with the status of the backend storage service.
[0035] The capacity control layer 20 specifically includes: a lossless scaling down execution module 203, a dual capacity verification module 202, an intelligent elastic scheduling module 204, a synchronization and disaster recovery rollback module 205, and a unified management module for scaling up and down 201. Specifically, the dual capacity verification module 202 is configured to receive and verify the capacity adjustment request automatically generated by the user or the intelligent elastic scheduling module 204 when the target management capacity is the target shrinkage capacity.
[0036] The dual capacity verification module 202 is a security barrier for capacity reduction operations. Through a dual verification mechanism of pre-verification after receiving a capacity reduction request and double confirmation before the capacity reduction operation is executed, combined with minimum capacity limits, it eliminates security risks such as data overflow and excessive capacity reduction from the source.
[0037] The lossless scaling-down execution module 203 is configured to determine the dedicated storage directory of the persistent volume declaration based on the identification information when the target management capacity is the target scaling-down capacity. The dedicated storage directory is used to uniquely identify the actual storage location of the target persistent volume in the corresponding target storage service, and the dedicated storage directory is deployed in the target storage service. The lossless scaling-down execution module 203 is also configured to call the target storage service's directory configuration interface through the first standard interface to adjust the quota information associated with the dedicated storage directory to the target scaling-down capacity. The first standard interface is a unified interface configured by the management system for multi-source heterogeneous storage services.
[0038] Specifically, the lossless scaling down execution module 203 is the core execution unit for the scaling down operation. It is responsible for adjusting the underlying capacity without affecting the business. During the scaling down process, it does not trigger any Pod restart, rebuild or scheduling operations, ensuring that the read and write operations of the business Pods are uninterrupted and imperceptible throughout the entire process.
[0039] The intelligent elastic scheduling module 204 is configured to periodically detect the current capacity utilization of the target persistent volume based on a preset time interval; The intelligent elastic scheduling module 204 is also configured to generate a second capacity adjustment request based on the current capacity utilization rate. The second capacity adjustment request includes identification information and target management capacity.
[0040] The synchronization and disaster recovery rollback module 205 is configured to be automatically triggered after expansion or contraction is completed, updating core fields such as storage.quota.actual in PVCAnnotations to synchronize the actual capacity status after expansion or contraction.
[0041] This application uses the same set of PVC Annotations fields for both expansion and contraction operations. The actual status of all capacity operations is synchronized to the Kubernetes management layer 10 through Annotations, achieving a strict closed loop of "expansion-contraction-status synchronization" and forming a unified storage capacity management system.
[0042] Storage adaptation layer 30: This is a storage interface abstraction layer that abstracts three common interfaces: query used capacity (query_used), set directory quota (set_quota), and get current quota (get_quota). It shields the interface differences between different storage systems and enables standardized interaction with the underlying storage layer 40. Among them, setting the directory quota (set_quota) is the first standard interface mentioned above, which is used to call the directory configuration interface of the target storage service to complete the storage quota modification operation.
[0043] The query_used() function is used to query the used capacity of a storage directory, and the get_quota() function is used to retrieve the current quota of a storage directory. These three general-purpose interfaces adopt a standardized design and are independent of specific storage systems.
[0044] Furthermore, the storage adaptation layer 30 also converts general interface requests into native calling methods that can be recognized by the corresponding storage system for storage systems that support directory quotas, such as JuiceFS, CephFS, and NFS: for JuiceFS, it calls native commands such as juicefs quota get and juicefs quota set to complete the corresponding functions of the interface; for CephFS, it calls its native API to implement interface capabilities; and for NFS, it uses the system's native quota management commands to perform operations.
[0045] This implementation method ultimately shields the differences in calling methods and command specifications between different storage systems. When a new storage system needs to be integrated in the future, only the underlying calling logic of the above three common interfaces needs to be implemented independently for that system, without adjusting the overall core scaling-down business process, effectively ensuring the scalability of the entire architecture.
[0046] Storage layer 40 is the actual storage resource layer of the cluster, supporting all general storage services with directory quota functions such as JuiceFS, CephFS, and NFS. No customization of storage services is required, and quota adjustment instructions can be directly received from storage adaptation layer 30.
[0047] In one possible implementation, the unified scaling management module 201 is also configured to monitor and manage the execution process of capacity adjustment requests, such as whether a disaster recovery rollback mechanism needs to be enabled after scaling down or scaling up.
[0048] like Figure 2 As shown, this application also provides a persistent volume capacity adjustment method, applied to a K8s cluster management system, the method comprising: S201. The management system determines the target management capacity of the target persistent volume and the identification information of the persistent volume declaration corresponding to the target persistent volume; One possible implementation involves the management system determining the identification information and target managed capacity of the persistent volume declaration, specifically: The management system receives a user's first capacity adjustment request, which includes identification information and the target management capacity; or Periodically check the current capacity utilization of persistent volume claims based on preset time intervals; A second capacity adjustment request is generated based on the current capacity utilization rate. The second capacity adjustment request includes identification information and target management capacity.
[0049] In other words, this application supports two ways to trigger capacity adjustment requests. The first is a user-initiated triggering method, in which the management system receives a first capacity adjustment request initiated by the user. This first capacity adjustment request includes parameters such as the namespace of the PVC, the PVC name, and the target management capacity. The second is an automatic system triggering method, in which the management system periodically checks the current capacity utilization of the target persistent volume in the cluster at configured time intervals (e.g., 5 minutes). Only when the current capacity utilization meets a certain condition for multiple consecutive detection cycles will a second capacity adjustment request be generated.
[0050] It is worth mentioning that the capacity adjustment request can be either a shrinking request or an expansion request. Shrinking requests are received and parsed by the dual capacity verification module 202; expansion requests are received and parsed by the Kubernetes management layer 10.
[0051] One possible implementation is to generate a second capacity adjustment request based on the current capacity utilization rate, specifically as follows: If the current capacity utilization rate of a consecutive preset number of units is less than or equal to the first threshold, then the target management capacity is set to the first preset value, and a first-level reduction request is generated based on the identification information and the first preset value. If the current capacity utilization rate of a consecutive preset number of units is greater than the first threshold and less than or equal to the second threshold, then the target management capacity is set to the second preset value, and a secondary reduction request is generated based on the identification information and the second preset value. If the current capacity utilization rate of a consecutive preset number of units is greater than or equal to the third threshold and less than the fourth threshold, then the target management capacity is set to the third preset value, and a first-level expansion request is generated based on the identification information and the third preset value. If the current capacity utilization rate of a consecutive preset number of units is greater than or equal to the fourth threshold, the target management capacity is set to the fourth preset value, and a secondary expansion request is generated based on the identification information and the fourth preset value, wherein the first threshold < the second threshold < the third threshold < the fourth threshold.
[0052] Specifically, the current capacity utilization rate = (used capacity / current quota information associated with the dedicated storage directory). For ease of description, the conditions for automatically generating the second capacity adjustment request in this application are referred to as the water level gradient conditions, which are pre-configured multi-level triggering rule sets based on the current capacity utilization rate of the target persistent volume.
[0053] Furthermore, this application supports both default water level gradient configuration and user-defined water level gradient configuration, with each threshold set based on the current capacity utilization of the target persistent volume.
[0054] For example, the default water level gradient configuration is as follows: Capacitance gradient: If the utilization rate is ≤30% (for two consecutive detection cycles), a level 1 capacity reduction will be triggered to increase the current capacity utilization rate to 50%. If the utilization rate is between 30% and 50% (for two consecutive detection cycles), a secondary capacity reduction will be triggered to increase the current capacity utilization rate to 70%. Capacity expansion gradient: If the utilization rate is between 80% and 90% (for two consecutive detection cycles), a level 1 capacity expansion is triggered, increasing the quota using the native Kubernetes process to reduce the current capacity utilization rate to 60%. If the utilization rate is ≥90% (for two consecutive detection cycles), a secondary capacity expansion will be triggered, increasing the quota using the native Kubernetes process to reduce the current capacity utilization rate to 70%. This application realizes intelligent automatic expansion and contraction of target persistent volumes. Specifically, this application automatically triggers expansion and contraction operations through water level gradient conditions and continuous detection cycle limits, without manual intervention. This not only adapts to the dynamic changes in business needs, but also avoids frequent capacity updates caused by small fluctuations in storage utilization, thus reducing the impact on the system and business.
[0055] It is worth mentioning that the subsequent steps of the capacity adjustment process automatically triggered by the system are completely consistent with the capacity adjustment process actively triggered by the user, requiring no manual intervention throughout the entire process, thus achieving intelligent and lossless capacity adjustment.
[0056] Example, Figure 4 The process of automatically generating a second capacity adjustment request for this application: S401, Configure water level gradient conditions; S402. Periodically check the current capacity utilization rate; S403, The current capacity utilization rate of a continuously preset number of units meets any of the following conditions; S404. Determine the type of gradient that is satisfied. If it is an expanding gradient, jump to steps S405~S406; if it is a shrinking gradient, jump to steps S407~S408. S405. Determine the target expansion capacity; S406, Generate expansion request; S407. Determine the target reduction capacity; S408, Generate a scaling-down request.
[0057] S202. If the target management capacity is the target shrinkage capacity, then: The dedicated storage directory for the persistent volume declaration is determined based on the identification information. The dedicated storage directory is used to uniquely identify the actual storage location of the target persistent volume in the corresponding target storage service, and the dedicated storage directory is deployed in the target storage service. S203. If the target management capacity is the target shrinkage capacity, the target storage service's directory configuration interface is called through the first standard interface to adjust the quota information associated with the dedicated storage directory to the target shrinkage capacity. The first standard interface is a unified interface configured by the management system for multi-source heterogeneous storage services.
[0058] Specifically, each PVC is uniquely matched with its dedicated storage directory. Based on the unique identifier (UID) of the PVC in Kubernetes, the dedicated storage directory (e.g., / pvc-{PVC-UID}) corresponding to the PVC in the underlying storage service is automatically matched to ensure the uniqueness of the storage directory and avoid accidental operation on other storage resources. Quota information represents the maximum available storage space limit set for a single dedicated storage directory by the underlying target storage service.
[0059] Specifically, by calling the set_quota interface of the underlying target storage service through the storage adaptation layer 30, the quota information of the exclusive storage directory is adjusted to the target shrink capacity. The shrink operation does not modify the core capacity configuration of the PVC (i.e., spec.resources.requests.storage), does not rebuild the PV / PVC, and does not remove the mounting relationship between the Pod and the PVC, thus achieving lossless shrinking of the target persistent volume.
[0060] This application utilizes an automatic shrinkage function to promptly reclaim idle storage capacity in the dedicated storage directory corresponding to the target persistent volume, avoiding resource waste caused by a large amount of storage space remaining in a low-utilization state for a long time. This can effectively reduce unnecessary storage device procurement expenditures and daily operation and maintenance management costs for enterprises, and has significant economic value.
[0061] Specifically, in this application, a unique correspondence is formed between the PVC and the underlying storage directory. Based on the unique identifier (UID) of the PVC in Kubernetes, the management system can automatically match the exclusive directory path corresponding to the PVC in the underlying storage system. An example path format is / pvc-{PVC-UID}, thus ensuring the uniqueness of the storage directory and preventing accidental modification of other storage resources during operation. This solution adopts a non-intrusive quota adjustment method. The storage adaptation layer 30 calls the set_quota interface of the underlying storage to adjust the quota of the aforementioned exclusive directory to the target capacity for scaling down. The entire process only changes the quota of the underlying storage directory; it does not modify the core capacity configuration of the PVC itself, nor does it require rebuilding PV and PVC resources, and it does not remove the mounting relationship between the Pod and the PVC.
[0062] One possible implementation, before determining the dedicated storage directory for the persistent volume claim based on the identification information, also includes: If the first capacity adjustment request / second capacity adjustment request is a scaling-down request, then determine whether the target management capacity in the first capacity adjustment request / second capacity adjustment request meets the preset rules. The preset rules are that the target management capacity is greater than or equal to the first real-time usage capacity and the target management capacity is greater than or equal to the preset minimum capacity. The minimum capacity is used to represent the minimum quota value of the target persistent volume in the target storage service. If not, the first capacity adjustment request / second capacity adjustment request is rejected and an error log is returned.
[0063] Specifically, the `query_used` interface of storage adaptation layer 30 is called to query the first real-time used capacity of the dedicated storage directory corresponding to the persistent volume declaration; the `get_quota` interface of storage adaptation layer 30 is called to query the current quota information of the dedicated storage directory currently in effect in the storage service, which is the maximum available storage space limit set by the storage service for the dedicated storage directory; and the `get_quota` interface of storage adaptation layer 30 is called to query the minimum capacity of the dedicated storage directory currently in effect in the storage service.
[0064] Furthermore, this application supports users to manually configure the minimum capacity, which is the lower limit for shrinking the target persistent volume. The target management capacity generated by any shrinking operation must not be lower than this value, which can effectively prevent insufficient storage space caused by excessive shrinking.
[0065] One possible implementation, after determining the dedicated storage directory for the persistent volume declaration based on the identification information, also includes: If the first capacity adjustment request / second capacity adjustment request is a reduction request, the target management capacity is verified again based on the second real-time usage capacity.
[0066] After retrieving the dedicated storage directory, this application calls the query_used interface again to query the second real-time usage capacity before performing quota adjustment. It confirms that there is no short-term surge in data during the two queries that causes the target management capacity to be less than the second real-time usage capacity. If the second confirmation fails, the scaling down operation is terminated immediately to avoid the risk of data overflow.
[0067] This application establishes a complete protection system through multiple verification mechanisms. The initial verification ensures that the target management capacity after scaling down is always greater than the real-time usage capacity, avoiding data overflow issues caused by scaling down. Before executing the scaling down operation, the legality of the scaling down request and the rationality of the target management capacity are verified again to prevent abnormal scaling down caused by misoperation. The minimum capacity mechanism allows users to set a lower limit for scaling down for each persistent volume. The target management capacity generated by the scaling down operation must not be lower than this value to avoid insufficient business storage space caused by excessive scaling down.
[0068] One possible implementation method also includes: Update the annotation information in the persistent volume declaration based on the target expansion or reduction capacity. The annotation information includes the quota information of the target persistent volume before reduction or expansion.
[0069] The annotation information is updated to prevent inconsistencies in state between the Kubernetes management layer 10 and the storage layer 40.
[0070] After the storage layer 40 returns a confirmation message that the quota adjustment was successful, this application automatically updates the annotations of the PVC and synchronizes the updated quota and other status information. One possible implementation method involves synchronizing the core information, including: ①storage.quota.actual: Quota information for the target persistent volume after update; ②storage.quota.used: The real-time used capacity of the target persistent volume; ③storage.quota.status: Status of shrinking / expanding operation (success / failure); ④ storage.quota.timestamp: Operation execution time; ⑤ storage.quota.minCapacity: Minimum capacity.
[0071] It's worth mentioning that operations and maintenance personnel can directly view the real status of the target persistent volume using native Kubernetes commands (such as kubectl describepvc), enabling a view between the Kubernetes management layer and the storage layer. Figure 1 To.
[0072] It is worth mentioning that the initial quota information of the target persistent volume in the target storage service in this application is obtained in the following way: When the management system is deployed and connected to the Kubernetes cluster for the first time, it will automatically call the Kubernetes native API to traverse all deployed existing persistent volume declarations in the cluster and obtain the unique identifier UID and the declared capacity of each persistent volume declaration; then, the system will automatically match its corresponding dedicated storage directory in the underlying storage system according to the unique identifier UID of the persistent volume declaration and according to the preset path rules (the path format is / pvc-{PVC-UID}); after the matching is completed, the system will automatically call the standardized set_quota interface of the storage adaptation layer 30 to write and make effective the original declared capacity of the persistent volume declaration as the initial quota value to the corresponding dedicated directory of the underlying storage system; finally, the system will automatically incorporate the persistent volume declaration into the unified capacity monitoring and automatic scaling management system.
[0073] In this application, the initial quota serves the following purpose: 1. Determine the trigger threshold when automatically generating a second capacity request for the first time.
[0074] 2. When existing PVCs are added to the K8s cluster management system of this application, they are used as "initial quotas"; 3. Used as a benchmark for subsequent annotation synchronization, that is, the K8s cluster management system 1 of this application needs to know the initial capacity in order to carry out subsequent management; 4. As a reference point for disaster recovery rollback: If problems occur during subsequent scaling down, it can serve as the original quota information to be restored.
[0075] This solution has good compatibility. For existing persistent volume declarations already deployed in the cluster, no manual modification is required. The system can automatically reuse the declared capacity of the persistent volume declaration as the initial quota, effectively reducing the migration cost of the solution.
[0076] One possible implementation method also includes: If the update fails, the disaster recovery rollback mechanism will be triggered to restore the quota information; Record disaster recovery rollback operations and send abnormal warnings.
[0077] Specifically, during scaling down or scaling up operations, if the underlying storage quota is successfully adjusted, but the PVC Annotations update fails due to network anomalies, insufficient permissions, or other reasons, the disaster recovery rollback mechanism is immediately triggered. The specific details of the disaster recovery rollback mechanism are as follows: The management system automatically retrieves the original quota information before the reduction operation; By calling the set_quota interface through storage adaptation layer 30, the quota of the dedicated storage directory is restored to the original quota information before the scaling down; Record rollback logs and send exception alerts to ensure that there are no inconsistencies in the state of storage layer 40 where the capacity has been modified and PVCAnnotations have not been synchronized.
[0078] This application achieves synchronization between PVC Annotations and storage quotas through Annotations synchronization and disaster recovery rollback mechanisms. When Annotations updates fail, they are automatically rolled back, ensuring that the actual capacity of the Kubernetes management layer 10 and the storage layer 40 is consistent. This allows operations and maintenance personnel to directly obtain the real capacity of the persistent volume through native Kubernetes commands without the need for underlying storage checks, significantly reducing operations and maintenance costs.
[0079] Furthermore, this application achieves capacity status synchronization through PVC Annotations, recording only the quota and other status information of the dedicated storage directory through this extended metadata field. The core capacity configuration of the persistent volume declaration will not be modified throughout the process, and no additional modifications are required to the existing cluster architecture, existing persistent volume declarations, and online services. The access method has non-intrusive characteristics, thereby effectively reducing the migration cost and deployment difficulty of this solution in actual business environments.
[0080] S204. If the target management capacity is the target expansion capacity, then modify the quota information of the persistent volume declaration based on the target expansion capacity.
[0081] It is worth mentioning that if the second capacity adjustment request is a shrinking request, it will be validated; if the second capacity adjustment request is a scaling-up request, the native Kubernetes scaling-up interface will be called directly, without any manual intervention.
[0082] Specifically, S204 modifies the spec.resources capacity configuration of the PVC to the target expansion capacity, and correspondingly, the capacity quota of the target storage service will also be modified to the target expansion capacity.
[0083] For example, when both the first capacity adjustment request and the second capacity adjustment request are scaling down requests, Figure 3 Here is another flowchart illustrating the persistent volume capacity adjustment method of this application: It should be noted that, Figure 3 The process branch starts from S301 or S304; S301~S303 and S304 are parallel branches. When either branch takes effect, the subsequent S305 step is executed to perform the capacity adjustment process.
[0084] S301. Periodically check the current capacity utilization rate; S302, The current capacity utilization rate of a continuously preset number of units meets the capacity reduction gradient; S303. Generate a second capacity adjustment request. The second capacity adjustment request includes the target reduction capacity and identification information. Proceed to step S305. S304. Receive a first capacity adjustment request initiated by the user. The first capacity adjustment request includes the target reduction capacity and identification information. S305, query the first real-time usage capacity and minimum capacity; S306, The target shrinkage capacity is greater than or equal to the first real-time usage capacity and the target shrinkage capacity is greater than or equal to the minimum capacity; S307. Determine the dedicated storage directory for the persistent volume declaration based on the identification information; S308. Verify the target reduced capacity again based on the second real-time usage capacity; S309. Call the set_quota interface to adjust quota information; S310, Log recording.
[0085] This application achieves unified management of expansion and contraction. Expansion operations follow the native Kubernetes process, while contraction operations are implemented by adjusting the quota information of the dedicated storage directory. Both types of operations share the same set of PVC Annotations to record capacity status, forming a complete management closed loop of expansion-contraction-status synchronization, which improves the management efficiency of storage capacity.
[0086] It should be noted that although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. On the contrary, the steps depicted in the flowchart may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.
[0087] On another front, this application also provides a computer-readable storage medium, which may be included in a computer device or exist independently, not assembled into the computer device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the methods described in this application. For example, it may execute... Figure 2 The steps of the method shown are as follows.
[0088] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0089] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.
Claims
1. A method for adjusting the capacity of a persistent volume, characterized in that, The method, applied to a Kubernetes cluster management system, includes: The management system determines the target management capacity of the target persistent volume and the identification information of the persistent volume declaration corresponding to the target persistent volume; If the target management capacity is the target shrinkage capacity, then: Based on the identification information, the dedicated storage directory of the persistent volume declaration is determined. The dedicated storage directory is used to uniquely identify the actual storage location of the target persistent volume in the corresponding target storage service. The target storage service's directory configuration interface is called through the first standard interface to adjust the quota information associated with the dedicated storage directory to the target shrinkage capacity. The first standard interface is the unified interface configured by the management system for the multi-source heterogeneous storage service. If the target management capacity is the target expansion capacity, then the quota information of the persistent volume declaration is modified based on the target expansion capacity and the identification information.
2. The method according to claim 1, characterized in that, The management system determines the target managed capacity of the target persistent volume and the identification information of the persistent volume declaration corresponding to the target persistent volume, specifically as follows: The management system receives a first capacity adjustment request from a user, the first capacity adjustment request including the identification information and the target management capacity; or Based on a preset time interval, the current capacity utilization of the target persistent volume is periodically detected; A second capacity adjustment request is generated based on the current capacity utilization rate. The second capacity adjustment request includes the identification information and the target management capacity.
3. The method according to claim 2, characterized in that, A second capacity adjustment request is generated based on the current capacity utilization rate, specifically as follows: If the current capacity utilization rate of a consecutive preset number of times is less than or equal to the first threshold, then the target management capacity is set to the first preset value, and a first-level reduction request is generated based on the identification information and the first preset value. If the current capacity utilization rate of a consecutive preset number of times is greater than the first threshold and less than or equal to the second threshold, then the target management capacity is set to the second preset value, and a secondary scaling-down request is generated based on the identification information and the second preset value. If the current capacity utilization rate of a consecutive preset number of times is greater than or equal to the third threshold and less than the fourth threshold, then the target management capacity is set to the third preset value, and a first-level expansion request is generated based on the identification information and the third preset value. If the current capacity utilization rate of a consecutive preset number of times is greater than or equal to the fourth threshold, the target management capacity is set to the fourth preset value, and a secondary expansion request is generated based on the identification information and the fourth preset value, wherein the first threshold < the second threshold < the third threshold < the fourth threshold.
4. The method according to claim 2, characterized in that, Before determining the dedicated storage directory of the persistent volume claim based on the identification information, the method further includes: If the first capacity adjustment request / second capacity adjustment request is a scaling-down request, then it is determined whether the target management capacity in the first capacity adjustment request / second capacity adjustment request meets the preset rule. The preset rule is that the target management capacity is greater than or equal to the first real-time usage capacity and the target management capacity is greater than or equal to the preset minimum capacity. The minimum capacity is used to characterize the minimum quota value of the target persistent volume in the target storage service. If not, then reject the first capacity adjustment request / second capacity adjustment request.
5. The method according to claim 4, characterized in that, After determining the dedicated storage directory for the persistent volume declaration based on the identification information, the method further includes: If the first capacity adjustment request / second capacity adjustment request is a capacity reduction request, the target management capacity is verified again based on the second real-time usage capacity. If the verification fails, the capacity reduction request is rejected.
6. The method according to claim 1, characterized in that, The method further includes: The annotation information in the persistent volume declaration is updated based on the target expansion capacity or the target reduction capacity. The annotation information includes the quota information of the target persistent volume before reduction or expansion.
7. The method according to claim 6, characterized in that, The method further includes: If the update fails, a disaster recovery rollback mechanism will be triggered to restore the capacity of the target persistent volume to its previous size before scaling down or scaling up.
8. A Kubernetes cluster management system, characterized in that, The management system includes: The dual capacity verification module is configured to determine the target managed capacity of the target persistent volume and the identification information of the persistent volume declaration corresponding to the target persistent volume; The lossless scaling-down execution module is configured to determine the dedicated storage directory of the persistent volume declaration based on the identification information when the target management capacity is the target scaling-down capacity. The dedicated storage directory is used to uniquely identify the actual storage location of the target persistent volume in the corresponding target storage service. The lossless scaling-down execution module is also configured to call the directory configuration interface of the target storage service through the first standard interface to adjust the quota information associated with the exclusive storage directory to the target scaling-down capacity. The first standard interface is the unified interface configured by the management system for the multi-source heterogeneous storage service. The Kubernetes management layer is configured to modify the quota information of the persistent volume declaration based on the target expansion capacity when the target managed capacity is the target expansion capacity.
9. The management system according to claim 8, characterized in that, The management system also includes: The intelligent elastic scheduling module is configured to periodically detect the current capacity utilization of the target persistent volume based on a preset time interval; The intelligent elastic scheduling module is also configured to generate a second capacity adjustment request based on the current capacity utilization rate, wherein the second capacity adjustment request includes the identification information and the target managed capacity; The dual capacity verification module is configured to receive a user's first capacity adjustment request or a second capacity adjustment request, wherein the first capacity adjustment request or the second capacity adjustment request includes the identification information and the target management capacity.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the persistent volume capacity adjustment method as described in any one of claims 1-7.