Cache quota management method and device for shared storage, medium and equipment

Abstract

The embodiment of the invention discloses a shared storage cache quota management method, which comprises the following steps of: on the basis of a quota updating request of a resource isolation unit, through the total amount of persistent data of the resource isolation unit on shared storage and a read cache proportion, combining with the number of copies corresponding to the resource isolation unit; and dynamically updating the quota upper limit cached by the resource isolation unit, and redistributing cache resources. By establishing a dynamic cache management strategy, the performance bottleneck in a write-intensive scene is solved. By dynamically sensing business load changes, the cache quota is automatically improved in the write-in peak period to guarantee performance, and the cost is controlled in the stationary period to avoid resource waste. The cache resource utilization rate is improved, and the operation cost is reduced.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, storage medium and device for managing cache quotas in shared storage. Background Technology

[0002] Shared cloud storage is an architecture that decouples data persistence from computing resources. Data is centrally hosted in object storage services provided by cloud service providers, while computing nodes such as database instances or application servers access this data in a shared manner via the network, thus decoupling the storage layer from the computing layer. Shared cloud storage is widely used in modern data-intensive systems. For example, big data analytics platforms use it as a unified data lake for concurrent processing by multiple computing clusters. Containerized applications and serverless FaaS also commonly rely on it as a persistent state backend to achieve elastic scaling for stateless computing.

[0003] In existing technologies, to address the high access latency of object storage service interfaces, compute nodes typically use local storage as a caching layer to retrieve frequently accessed data from remote object storage and temporarily store it locally. Each compute node provides business support to multiple tenants. To ensure stable business operation, a tenant is instantiated as one or more resource isolation units. These resource isolation units perform quota isolation and scheduling of physical resources such as local caches, thereby achieving performance and fault isolation between tenants and preventing resource contention.

[0004] However, existing technologies for cache allocation in computing nodes often employ fixed rules or simple proportional allocation methods. These strategies can only perform static allocation based on preset parameters or remaining resources, failing to recognize dynamic changes in actual business scenarios. For example, when data replicas migrate or are added between nodes, fixed cache allocation strategies cannot identify such topology changes and therefore cannot adjust the distribution of cache resources accordingly. Similarly, static strategies cannot respond promptly to large volumes of write data or sudden changes in data access patterns. It is difficult to effectively balance the conflict between storage costs and write performance while ensuring multi-tenant isolation.

[0005] Based on this, this specification provides a cache quota management method for shared storage to partially solve the problems existing in the prior art. Summary of the Invention

[0006] This specification provides a method, apparatus, storage medium, and electronic device for managing cache quotas in shared storage, in order to partially solve the problems existing in the prior art.

[0007] The embodiments in this specification adopt the following technical solutions: This specification provides a method for managing cache quotas in shared storage, which is applied to compute nodes. The method includes: Receive cache quota update requests from resource isolation units; Obtain the quota calculation parameters of the resource isolation unit, wherein the quota calculation parameters include at least: the total amount of persistent data on the shared storage and the read cache ratio; Based on the quota calculation parameters and the number of replicas corresponding to the resource isolation unit, the quota limit of the dynamic cache of the resource isolation unit on the computing node is re-determined; The cache resources allocated to the resource isolation unit are adjusted according to the quota limit.

[0008] This specification provides an apparatus for managing cache quotas in shared storage, the apparatus comprising: The receiving module is used to receive cache quota update requests from resource isolation units; The acquisition module is used to acquire the quota calculation parameters of the resource isolation unit, wherein the quota calculation parameters include at least: the total amount of persistent data on the shared storage and the read cache ratio; The quota update module is used to redetermine the upper limit of the dynamic cache quota of the resource isolation unit on the computing node based on the quota calculation parameters and the number of replicas corresponding to the resource isolation unit; The quota management module is used to adjust the cache resources allocated to the resource isolation unit according to the quota limit.

[0009] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for managing cache quotas in shared storage.

[0010] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method for managing the cache quota of the shared memory described above.

[0011] The above-described at least one technical solution used in the embodiments of this specification can achieve the following beneficial effects: This specification discloses a method for managing cache quotas in shared storage. Based on quota update requests from resource isolation units (ROUs), this method dynamically updates the cache quota limit of each ROU, reallocating cache resources, by considering the total persistent data volume, write cache ratio, and read cache ratio of the ROU in the shared storage, combined with the number of replicas corresponding to that ROU. By establishing a dynamic cache management strategy, it solves the performance bottleneck in write-intensive scenarios. By dynamically sensing changes in business load, it automatically increases the cache quota during peak write periods to ensure performance, and controls costs and avoids resource waste during stable periods. This improves cache resource utilization and reduces operating costs. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 A flowchart illustrating shared storage cache quota management provided in the embodiments of this specification; Figure 2 A schematic diagram illustrating the two-tier quota adjustment process provided in the embodiments of this specification; Figure 3 This is a schematic diagram of data migration provided for embodiments of this specification; Figure 4 This is a schematic diagram of data migration provided for embodiments of this specification; Figure 5 A schematic diagram of a shared storage cache quota management device provided for embodiments of this specification; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0014] In shared storage architectures where compute and storage are separated, such as cloud-native databases, compute nodes typically need to maintain local data caches to improve access performance to remote object storage. To ensure business stability among multiple tenants—resource-isolated units—and to provide a deterministic performance baseline for each tenant, compute nodes need to set upper and lower limits for local cache resources for each tenant. Traditional quota management strategies generally use fixed allocations or simple proportional models based on the amount of persistent data; for example, the upper limit is determined by the larger of a fixed upper limit and the amount of persistent data. The core problem with this type of static strategy is that its allocation logic is relatively fixed once set, making it difficult to adapt to dynamic changes in real business load.

[0015] First, in terms of performance and cost balance, static strategies cannot accurately respond to load fluctuations. During initial data writes, batch imports, or peak business periods, a large amount of data is temporarily stored as "dirty data" in the local write cache, while the amount of persistent data has not yet increased. This results in a severe shortage of computing quotas based on the latter, causing write blocking and performance jitter. Conversely, during off-peak business periods, static quotas will lead to idle and wasted cache resources, resulting in poor cost-effectiveness.

[0016] Secondly, static strategies lack the necessary flexibility in multi-tenant resource isolation and sharing. They cannot intelligently reallocate idle cache resources to tenants with urgent needs based on the real-time priority and urgency of each tenant's business, while ensuring the lower limit of their cache quota. This makes it difficult to achieve a balance between strict isolation and global resource utilization.

[0017] Furthermore, static strategies appear rigid and outdated when facing business changes. They fail to detect and quickly adapt to the continuous growth of total data volume, the natural shift of access hotspots, and changes in topology such as the number and location of replicas, leading to a sustained decline in service performance after data migration. For example, when data replicas are migrated to a new node, static strategies cannot proactively pre-warm up the cache on the new node.

[0018] Therefore, it is evident that cache quota allocation mechanisms based on fixed rules struggle to simultaneously achieve performance stability, cost-effectiveness, isolation fairness, and dynamic adaptability in the dynamic and complex scenario of shared storage multi-tenant databases. This specification proposes a cache quota management method for shared storage, achieving a better balance between performance, cost, and complexity. The technical solutions provided by the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0019] Figure 1 A flowchart for shared storage cache quota management provided in the embodiments of this specification specifically includes the following steps: S100: Receives a cache quota update request from a resource isolation unit.

[0020] In the embodiments described in this specification, the following are employed: Figure 1 The device managing cache quotas in shared storage can be any electronic device, such as a computer, server, or server cluster consisting of multiple servers. For ease of description, the following explanation uses a server as an example. This server is the computing unit in the shared storage scenario, that is, the device that retrieves data from object storage devices to provide tenants with the data and computing resources required for business execution. This computing unit allocates cache to each tenant based on its own cache space. In a shared storage architecture, cache quota refers to the logical resource limit allocated and managed by the compute node for the resource isolation unit running on it, used to temporarily store copies of data in local storage media (such as memory or solid-state drives). This cache quota defines the upper limit of local cache space that the resource isolation unit can use, aiming to accelerate access to data in remote shared storage by utilizing local storage to compensate for network access latency. The allocation, adjustment, and management of cache quotas are implemented by the compute node and are the core mechanism for achieving performance isolation and resource control in a multi-tenant environment.

[0021] To address the issue of rigid cache resource allocation in multi-tenant architectures, which struggles to dynamically adapt to changes in business load and data replica topology, this specification provides a write-aware dynamic cache quota management method. By combining the total amount of persistent data, the user-defined upper limit for the read cache ratio, and the number of replicas, the cache quota upper limit for each resource isolation unit (RUIU) is dynamically calculated. This method can automatically adjust quotas based on real-time characteristics of the business load, such as the total amount of persistent data in the RUIU, increasing quotas during peak write periods to ensure performance and tightening quotas during stable periods to control costs. Furthermore, through a multi-level collaborative adjustment mechanism, scaling decisions can be made at both the RUIU and compute node levels, achieving elastic scaling and efficient global utilization of cache resources while maintaining multi-tenant resource isolation and performance baselines, significantly improving resource utilization and reducing costs.

[0022] Specifically, the server receives cache quota update requests from resource isolation units.

[0023] In the embodiments described in this specification, the server continuously listens for and processes communication requests from various resource isolation units within its jurisdiction, including cache quota update requests. These requests are initiated by the resource isolation units, triggering the server to adjust the local cache resource quota allocated to those units. Typically, the resource isolation unit generates and sends cache quota update requests based on continuous monitoring and evaluation of its own cache resource usage.

[0024] Specifically, resource isolation units can periodically calculate their local cache utilization rate, i.e., the proportion of currently used cache space to their existing quota, based on background services or triggered by events. When this utilization rate exceeds a preset unit expansion threshold, the resource isolation unit determines that its cache space is strained, potentially affecting subsequent writes or access to hot data, and thus generates and sends an expansion update request. Conversely, when the utilization rate is below a preset unit shrinkage threshold, the resource isolation unit can determine that resources are idle, and to improve overall resource efficiency, it generates and sends a shrinkage update request. This threshold-based proactive request mechanism enables the server to respond promptly to changes in the actual needs of the resource isolation units.

[0025] Of course, in the embodiments described in this specification, after receiving the cache quota update request, the server may also perform operations such as verifying the legality and integrity of the request and generating a business identifier, which will not be described in detail here.

[0026] S102: Obtain the quota calculation parameters of the resource isolation unit, wherein the quota calculation parameters include at least the total amount of persistent data on the shared storage and the read cache ratio.

[0027] In the embodiments described in this specification, in response to a cache quota update request, the server can obtain the parameters required for quota calculation, preparing necessary and accurate input data for subsequent dynamic calculation of the cache quota limit. The quota calculation parameters are predefined or continuously maintained, collectively forming the rules and constraints framework for quota calculation.

[0028] Specifically, the server can obtain the total amount of persistent data for a resource isolation unit on shared storage. This total amount of data refers to the portion of all data in that unit that has been flushed and stably stored in the remote shared storage service. It represents the scale of data that the resource isolation unit has committed and is globally accessible. The server can obtain this value directly by querying the shared storage's metadata service interface, or by accessing its own maintained data volume statistics table.

[0029] For example, for a database tenant storing customer profiles, the total amount of persistent data is the total size of all customer records successfully stored in object storage, and this value will continue to grow as new customers register or profiles are archived.

[0030] Additionally, the server can obtain the read cache ratio of the resource isolation unit to constrain the allocation ratio of read cache resources based on persistent data. The server can read this parameter from a unified configuration management center or a local policy configuration file.

[0031] In addition, different read caching ratios can be configured for different business scenarios. For example, an analytical tenant that mainly handles complex report queries may be given a higher read caching ratio in order to keep more frequently queried data locally.

[0032] After successfully obtaining the aforementioned core parameters, the server completed the preparation of the key information required for quota calculation.

[0033] S104: Based on the quota calculation parameters and the number of replicas corresponding to the resource isolation unit, redetermine the upper limit of the dynamic cache quota of the resource isolation unit on the computing node.

[0034] In the embodiments described in this specification, after obtaining the quota calculation parameters, the server can calculate the upper limit of the dynamic cache quota for the resource isolation unit on the computing node. The calculation of the upper limit of the quota comprehensively considers multiple dimensions of factors, including the data scale of the resource isolation unit, real-time load characteristics, user cost strategies, and system deployment topology. Among them, the total amount of persistent data reflects the basic data scale of the unit and is the fundamental basis for allocating cache resources. At the same time, the total amount of persistent data and the number of replicas also reflect the business load of the resource isolation unit. Considering the number of replicas is to solve the cache redundancy problem under multi-replica deployment. By distributing the total cache budget among replicas, the global resource utilization is optimized and unnecessary storage costs are reduced. The dynamic quota upper limit determined by these factors enables the system to intelligently adapt to load changes and topology changes while ensuring the lower limit of the cache quota for each resource isolation unit, achieving a balance between performance and cost.

[0035] Specifically, after obtaining the complete quota calculation parameters, the server can perform a comprehensive calculation based on these parameters. The server can first calculate the read cache requirement budget for this resource isolation unit based on the total amount of persistent data and the read cache ratio. This read cache requirement budget represents the recommended cache quota to be allocated to accelerate access to all persistent data in this resource isolation unit.

[0036] The server then calculates the read cache demand budget and, combined with the total number of replicas, redetermines the quota limit. In other words, the read cache demand budget is divided by the total number of replicas to obtain the theoretical quota value distributed across multiple replicas.

[0037] Finally, the server compares this allocated theoretical quota value with the minimum cache quota guaranteed for the resource isolation unit, and takes the larger of the two as the final redefined dynamic cache quota limit. Thus, the quota limit determined by the server can be determined based on the principles of data proportion allocation and replica cost amortization, while also satisfying the most basic resource needs of the unit at lower levels.

[0038] As in the formula: Here, the total amount of persistent data is S, the number of replicas is Z, the read cache ratio is m, and the lower limit of the cache quota is L. S×m reflects the cost control of resource isolation units, and the cache resources allowed are proportional to the total amount of data, which can be adjusted by the user through the ratio coefficient m. The larger the data volume, the higher the cache quota, but it is always constrained by the read cache ratio. Z reflects the cost amortization optimization in multi-replica scenarios.

[0039] Alternatively, in one or more embodiments of this specification, the quota calculation parameters may further include a write cache ratio. The user-defined read / write cache ratio reflects a cost control strategy based on business characteristics, such as read-intensive or write-intensive operations, allowing for differentiated allocation of cache resources. The server can also determine the corresponding write cache requirement budget based on the write cache ratio and the total amount of persistent data. Then, a competition is held between the read cache budget and the write cache budget, and the larger value is used as the basis for the allocation quota.

[0040] Then, by combining the total number of replicas and the minimum cache quota, the upper limit of the dynamic cache quota is determined.

[0041] As in the formula, Here, the write cache ratio is n. m and n reflect the impact of considering different business types on cache quotas. When the resource isolation unit supports read-intensive business, the user can set m higher, and S×m will dominate the budget, increasing the amount of hot data residing in the cache to accelerate reads. Conversely, when the business is write-intensive, setting n higher will allow S×n to dominate the budget, reserving sufficient space for the write buffer to avoid blocking.

[0042] Specifically, the server retrieves the preset write cache ratio for this resource isolation unit. This write cache ratio can be set by the system administrator or by the user based on cost and performance expectations. This write cache ratio defines the upper limit of the ratio between the cache space allocated to this resource isolation unit and its total data volume, specifically used to constrain write cache resources. The server reads this parameter from a unified configuration management center or a local policy configuration file.

[0043] For example, for a core transaction tenant that is extremely sensitive to write latency, the administrator may set a high limit on its write cache ratio to ensure that its burst write traffic can be adequately buffered.

[0044] S106: Adjust the cache resources allocated to the resource isolation unit according to the quota limit.

[0045] In the embodiments described in this specification, after the quota limit is determined, the server can adjust the cache resources.

[0046] Specifically, the server can first determine the current quota of the resource isolation unit. The current quota refers to the maximum amount of local cached resources that the server actually allocates to that resource isolation unit and allows it to use at that moment. The server obtains this accurate value by querying its maintained resource quota allocation table or the resource management data structure in memory.

[0047] Next, the server performs a numerical comparison operation to determine whether the quota limit is greater than the current quota. This comparison result is the core basis for determining the direction of subsequent adjustments.

[0048] When the comparison result shows that the quota limit is greater than the current quota, the server determines that the cache resources for this resource isolation unit need to be expanded. At this time, the server will perform the operation of increasing the cache resources allocated to the resource isolation unit based on the quota limit. The server will send a resource grant instruction to the resource isolation unit and update the internal quota record to ensure that the resource isolation unit can obtain new cache resources not exceeding the quota limit.

[0049] For example, during the midday business peak, calculations show that a certain analytical tenant's quota limit should be increased to 150GB, higher than its current 100GB quota. The server then initiates a process to allocate an additional 50GB of cache space for it.

[0050] When the comparison result shows that the quota limit is not greater than the current quota, the server determines that the existing resource allocation needs to be reviewed and adjusted based on the new limit. At this point, the server will adjust the cache resources allocated to resource isolation units based on the quota limit. This adjustment usually means reducing the allocation to meet a stricter limit, or maintaining the existing allocation if the quota limit equals the current quota.

[0051] For example, during off-peak hours at night, if a tenant's quota limit is calculated to be 80GB, which is lower than their current 100GB quota, the server will trigger a process to reclaim 20GB of their surplus cache resources for reallocation. If their current quota is 80GB, no adjustment will be made.

[0052] based on Figure 1 The method described here involves managing cache quotas for shared storage. Based on quota update requests from resource isolation units (RIPs), this method dynamically updates the cache quota limit for each RIP based on its total persistent data volume, write cache ratio, and read cache ratio, combined with the number of replicas corresponding to that RIP. It then reallocates cache resources accordingly. By establishing a dynamic cache management strategy, this method addresses performance bottlenecks in write-intensive scenarios. By dynamically sensing changes in business load, it automatically increases cache quotas during peak write periods to ensure performance, while controlling costs and avoiding resource waste during stable periods. This improves cache resource utilization and reduces operating costs.

[0053] In addition, in one or more embodiments of this specification, besides considering the total amount of persistent data of the resource isolation unit in shared storage, the server may also consider the amount of data of the resource isolation unit locally, that is, the non-persistent data. This non-persistent data is treated as dirty data; that is, this dirty data does not need to be persisted to shared storage immediately, but can be stored locally on the server first, and then updated to the nodes in shared storage according to business logic or persistence strategy. However, this dirty data will also consume the cache resources of the resource isolation unit, so the impact of dirty data should be further considered.

[0054] The dirty data volume refers to the total amount of data that has not yet been persisted to remote shared storage through background tasks. The dirty data volume is generally obtained by the resource isolation unit by real-time statistics of the dirty page status in its local memory or solid-state drive before initiating a request.

[0055] Therefore, in step S100, the cache quota update request may also carry the amount of dirty data of the resource isolation unit on the current server.

[0056] In step S104, the server may also consider dirty data. Specifically, after obtaining the complete quota calculation parameters, the server performs the calculation based on the obtained parameters and the amount of dirty data carried in the request.

[0057] First, the server can integrate inputs, including: the total amount of persistent data representing the data scale, the amount of dirty data, and the write cache and read cache ratios preset by the user to control resource allocation. Simultaneously, the server needs to confirm the total number of replicas of the current data set within the resource isolation unit in the distributed environment. The total number of replicas is dynamic system topology information, and the server obtains new values ​​by querying the cluster's metadata management service. For example, a product database tenant that temporarily adds two read-only replicas to cope with the "Black Friday" shopping peak will see its total number of replicas increase from three to five.

[0058] The server can weight and combine the total amount of persistent data with the amount of dirty data to form a composite metric that reflects the "current amount of data of concern," i.e., the data volume ceiling. Subsequently, the server can multiply the read cache ratio by the data volume ceiling to obtain an estimated total cache budget.

[0059] Then, considering that the same data has cache redundancy across multiple replicas, in order to optimize costs, the server divides the estimated total cache budget by the current total number of replicas of the resource isolation unit, thereby distributing the budget among the replicas.

[0060] Finally, the server compares the calculated allocated budget value with the minimum cache quota required by the resource isolation unit specification. The server takes the larger of the two values ​​as the redefined dynamic cache quota upper limit. This ensures that the resource isolation unit receives at least the basic cache resources promised in its service level agreement.

[0061] For example, even if the calculated allocated budget is low, the resource isolation unit can still be guaranteed its cache quota lower limit. At this point, the server has completed a dynamic quota upper limit redetering process, and this upper limit will serve as the direct basis for subsequent resource adjustment operations.

[0062] In the embodiments described in this specification, the server can also first calculate the read cache requirement budget and the write cache requirement budget. By introducing a real-time write load metric, i.e., the amount of dirty data, the server can compete and trade off between the read cache budget for accelerating historical data and the write cache budget for real-time write services. Instead of pre-assuming whether the business is read-heavy or write-heavy, it automatically selects the budget that best suits the current business characteristics based on the actual amount of persistent data and dirty data.

[0063] For example, the server can weight and combine the total amount of persistent data with the amount of dirty data to form a composite metric reflecting the "currently important total amount of data," i.e., the data volume ceiling. Then, the server can derive a comprehensive cache ratio based on the write cache ratio and the read cache ratio, for example, taking the maximum or average of the two as the overall resource allocation coefficient. Next, the server multiplies the composite metric by this comprehensive cache ratio to obtain a theoretical total cache budget estimate. Finally, the final cache quota is determined through allocation.

[0064] Alternatively, in the embodiments described in this specification, the server may first determine a read cache requirement budget based on the total amount of persistent data.

[0065] The server can multiply the total amount of persistent data obtained by the read cache ratio; the product is the read cache requirement budget. This budget represents the upper limit of local cache space recommended for allocation under ideal conditions to accelerate access to all persistent data in this resource isolation unit. For example, for a report analysis tenant with a total persistent data volume of 1000GB and a read cache ratio set to 20%, the server calculates a read cache requirement budget of 200GB, which provides a quantitative reference target for storing the hot data involved in its high-frequency queries.

[0066] Furthermore, the server can also determine the write cache requirement budget for this resource isolation unit on the compute node.

[0067] The calculation of this write cache requirement budget integrates data scale and real-time load. The server first adds the total amount of persistent data to the amount of dirty data carried in the request, obtaining a value reflecting the total amount of data that can be processed, i.e., the data volume limit. Then, the server multiplies the data volume limit by the write cache ratio limit to obtain the write cache requirement budget. This write cache requirement budget quantifies the cache space required to effectively buffer current and potential write operations. For example, during data import, the amount of dirty data for the same tenant may surge to 500GB, while the total amount of active data is 1500GB. If the write cache ratio is 30%, the write cache requirement budget is 450GB, which is much higher than its read cache requirement budget, accurately capturing the resource demand during peak write periods.

[0068] Finally, the server allocates the dynamic cache quota limit based on the read cache demand budget and the write cache demand budget, and according to the number of replicas corresponding to the resource isolation unit.

[0069] In the embodiments described in this specification, the dynamic quota limit can be calculated using the following formula: The total amount of persistent data is S, the amount of dirty data is X, the number of replicas is Z, the read cache ratio is m, the write cache ratio is n, and the lower limit of the cache quota is L. This reflects the competition between read and write budgets, prioritizing the more pressing resource needs. Z represents cost allocation optimization in multi-replica scenarios. The entire formula dynamically unifies load awareness, cost control, and performance isolation.

[0070] Furthermore, in addition to non-persistent data, the server can also consider the cache quota requirements of temporary data in resource isolation units. Dirty data may also include operation records in the write cache, log buffers, or intermediate transaction states.

[0071] In step S104, the server may consider only temporary dirty data, or it may consider both the dirty data to be persisted and the temporary dirty data. Regardless of the approach, the amount of dirty data reflects the current write pressure and data backlog of the resource isolation unit in real time. This is crucial for handling scenarios such as sudden writes and batch imports, and can prevent write blocking caused by insufficient quota.

[0072] When only temporary dirty data is considered, this write cache requirement budget represents a value reflecting the total amount of currently active data. If all dirty data awaiting persistence and temporary dirty data are considered, the "budget" range is further expanded, further preventing write blocking after adjusting cache quotas.

[0073] Based on the amount of dirty data in the quota update requests of a resource isolation unit (ROU), as well as the total amount of persistent data, write cache ratio, and read cache ratio of the ROU on shared storage, and combined with the number of replicas corresponding to that ROU, the quota limit of the ROU's cache is dynamically updated, and cache resources are reallocated. By introducing the amount of unflushd dirty data as a quantitative indicator of real-time write load, and establishing a dual-budget contention mechanism in conjunction with the amount of persistent data, the performance bottleneck in write-intensive scenarios is resolved. By dynamically sensing changes in business load, the cache quota is automatically increased during peak write periods to ensure performance, while costs are controlled and resource waste is avoided during stable periods. Furthermore, considering the number of replicas to distribute the cache budget improves cache resource utilization and reduces operating costs.

[0074] In one or more embodiments of this specification, when the server adjusts cache resources in step S106, it may perform expansion or contraction based on the request type explicitly specified in the cache quota update request.

[0075] Specifically, the server first parses the cache quota update request to identify the request type declared by the resource isolation unit. The request type is a specific request proactively made by the resource isolation unit based on its assessment of its own cache usage. For example, a tenant performing a large-scale data import, sensing that its local cache space is about to run out, will explicitly declare this as a capacity expansion request in the request. The server obtains this type information by parsing specific fields in the request message.

[0076] When the server identifies the current request as a capacity expansion / update request, it initiates a process to increase cache resources. In this process, the server uses the recalculated dynamic cache quota limit as the upper limit for resource increase operations. First, it determines whether the cache resources currently allocated to this resource isolation unit have reached this quota limit. If not, the server increases the allocated cache resources based on its existing quota until the quota limit is reached.

[0077] For example, if the server calculates a new quota limit of 200GB for a tenant, and the tenant's current quota is 150GB, then after receiving the tenant's expansion update request, the server will allocate an additional 50GB of cache resources for the tenant.

[0078] When the server identifies the current request as a scaling-down update request, it will initiate a process to reduce cached resources. In this process, the server uses the recalculated dynamic cache quota upper limit as a reference target for resource reclamation, but must simultaneously ensure that the cache quota held by the resource isolation unit after scaling down is not lower than its guaranteed cache quota lower limit.

[0079] The server can compare the current quota of the resource isolation unit with the calculated quota limit and the cache quota limit. If the current quota is higher than the quota limit, the server can reduce the quota to the larger of the quota limit and the cache quota limit.

[0080] For example, if a tenant's current quota is 100GB, the newly calculated quota limit is 60GB, and their cache quota limit is 30GB, then upon receiving their scaling-down update request, the server will reduce their quota to 60GB. Furthermore, in one or more embodiments of this specification, the resource isolation unit has the ability to autonomously monitor the usage of local cache resources. Thus, the resource isolation unit can obtain its current cache utilization rate through periodic sampling and calculation. This cache utilization rate refers to the ratio between the actual cache space occupied by the resource isolation unit and its currently allocated cache quota.

[0081] The resource isolation unit compares the calculated cache utilization rate with the pre-configured policy threshold and decides whether to generate and what type of cache quota update request to generate based on this.

[0082] Specifically, in the embodiments described in this specification, the autonomous monitoring and request generation capabilities of the resource isolation unit can be achieved through a lightweight background service process deployed in its operating environment. This background service, as a resident task within the resource isolation unit, operates independently of its main business logic and is specifically responsible for the self-observation and elastic management of resources.

[0083] The background service periodically wakes up and executes its monitoring tasks. Within a monitoring cycle, it first obtains the current cache space size of the resource isolation unit by calling the local cache management interface or reading the system statistics file. At the same time, it queries the currently allocated cache quota value from the communication module with the server or the local configuration. Subsequently, the background service calculates the accurate cache utilization rate based on these two values.

[0084] After completing the calculation, the backend service compares the resulting cache utilization rate with the policy thresholds preloaded into its memory. These policy thresholds, namely the unit shrinking threshold and the unit expanding threshold, are predefined by the tenant's user and issued when the resource isolation unit is initialized.

[0085] If the cache utilization rate is lower than the unit scaling-down threshold, the backend service assembles a scaling-down update request indicating the intention to scale down; if the cache utilization rate is higher than the unit scaling-up threshold, it assembles a scaling-up update request indicating the intention to scale up. This request message includes parameters such as the request type, resource isolation unit identifier, and the amount of unflushed dirty data.

[0086] Finally, the background service asynchronously sends the assembled cache quota update request through its communication link to its associated computing node, i.e., the server. After sending, the background service enters a dormant state, waiting for the next monitoring cycle. This design ensures that the requests of resource isolation units can be reported in a timely and automatic manner without the need for their core business processes to be aware of or interrupted, thus achieving automation and decoupling of resource management.

[0087] Furthermore, in the embodiments of this specification, in step S106, if the server determines that it needs to add cache resources to the resource isolation unit, it may not immediately perform the allocation, but instead first conduct a round of global resource review and planning at the node level. This ensures that the expansion will not exceed the physical carrying capacity of the computing node itself, or that node expansion will be actively triggered when resources are insufficient, thereby ensuring the feasibility of the adjustment operation.

[0088] First, the server can determine a total expected cache resource amount based on the newly calculated quota limit for this resource isolation unit and the current quotas of all other resource isolation units in the compute node. This expected cache resource amount is the total amount of cache resources the entire compute node is expected to support after this adjustment. For example, on a compute node running five tenants, if the server plans to increase tenant A's quota limit from 100GB to 150GB, while the combined current quotas of the other four tenants are 400GB, then the server calculates a total expected cache resource amount of 550GB.

[0089] The server can then determine whether the actual cache resources currently available to the server point meet the expected cache resources. It compares the total physical space currently available for caching on local storage devices, such as solid-state drives, with the calculated expected cache resources.

[0090] If the current total physical space is greater than or equal to the expected cached resources, the server determines that the condition is met.

[0091] When the server determines that the current cached resources of a node meet expectations, it directly executes the operation of increasing the cached resources allocated to the resource isolation unit based on the quota limit. The server can allocate the additional cache space to this resource isolation unit through the internal resource management interface and update the quota record.

[0092] If the server determines that the current cached resources of a node do not meet expectations, it means that the node's existing physical space is insufficient to support this adjustment. In this case, the server can first determine the resource shortfall based on the quota limit and trigger the process of expanding the cached resources of the compute node itself.

[0093] The server can call the expansion interface of the underlying infrastructure, such as requesting the cloud platform to increase the capacity of the local cloud disk. After the cache resource expansion operation of the compute node is completed and confirmed, the server will then execute the operation of increasing the cache resources allocated to the resource isolation unit based on the quota limit.

[0094] Similarly, in the embodiments of this specification, in step S106, if the server determines that it needs to reduce cache resources for the resource isolation unit, the server's operation follows a two-level decision logic: first the resource isolation unit, then the compute node. This logic first ensures that the quota of the resource isolation unit itself meets the new upper limit constraint, and then evaluates from a global perspective whether the physical resources of the compute node can be reclaimed.

[0095] The server can first perform an operation to reduce the cache resources allocated to a resource isolation unit. This operation is triggered when the server determines that the quota limit is less than the current quota of that resource isolation unit. The server calculates the difference between the current quota and the quota limit to determine the amount of cache resources that need to be reclaimed.

[0096] Subsequently, the server can notify the resource isolation unit through the resource management interface to release the corresponding amount of cache space, or directly reclaim its allocated but unused cache quota. For example, if the server calculates and finds that a tenant's quota limit has been updated to 80GB, while its current quota is 100GB, the server will initiate a process to reclaim 20GB of cache resources from that tenant.

[0097] Then, after reducing the cache resources of the resource isolation unit, the server assesses the impact of this operation on the overall resource utilization of the compute nodes. The server can determine whether the current cache utilization of the compute nodes is lower than the preset node scaling-down threshold.

[0098] The server can aggregate the total amount of cache currently used across all resource isolation units on the compute node and compare it with the total available cache capacity of the compute node to determine the current cache utilization rate. The node scaling-down threshold is a pre-configured policy value used to determine whether node resources are in an inefficient or idle state.

[0099] Finally, if the server determines that the current cache utilization of a compute node is lower than the preset node scaling-down threshold, it indicates that the compute node has significant resource surplus. In this case, the server will trigger a scaling-down operation for the compute node itself. This operation may include sending a request to the underlying infrastructure to reduce the local storage capacity allocated to this node, thereby reducing resource holding costs.

[0100] If the current cache utilization rate of a compute node is not lower than the preset node scaling-down threshold, no cache changes will be made to the compute node.

[0101] Figure 2This is a flowchart illustrating the two-level quota adjustment process provided in this manual. The overall process begins with the server receiving a cache quota update request from a resource isolation unit (RISU). The server first parses and determines the type of the request. If the request is for expansion, the process proceeds to the left branch: the server first assesses whether the available resources of the compute node are sufficient; if sufficient, it directly performs an expansion operation at the RISU level, updates its quota record, and ends the process; if the node resources are insufficient, it first triggers expansion at the compute node level, and after the node expansion is completed, it performs quota increases at the RISU level. If the request is for reduction, the process proceeds to the right branch: the server first performs a reduction operation at the RISU level; after completion, it further determines whether the overall cache utilization of the compute node is lower than a preset node reduction threshold; if lower, it triggers a reduction operation at the compute node level; if not lower, the process ends directly. This process clearly demonstrates the collaborative decision-making and linkage mechanism of quota adjustment between the RISU and compute node levels.

[0102] In the embodiments of this specification, the multi-tenant architecture of shared storage also includes scenarios involving the deletion of resource isolation units. The deletion of a resource isolation unit is not merely a local resource release event, but also a crucial opportunity to trigger a global system resource review and cost optimization.

[0103] Specifically, the server handles not only dynamic quota adjustment requests for resource isolation units (RIUs), but also their lifecycle termination events. When the server receives a delete command for a specific RUI, it means that the RUI will be destroyed, and all its resources need to be thoroughly cleaned up. This command may be manually triggered by the system administrator or automatically issued by a higher-level resource orchestration system based on business policies.

[0104] In response to a resource isolation unit (RIU) deletion command, the server reclaims the RIU's current cache quota. The server identifies the RIU based on the deletion command and performs the operation to reclaim its current cache quota. This operation involves two aspects: first, the server notifies the RIU to immediately cease use and release all occupied local cache space. Simultaneously, the server marks the cache quota originally allocated to the RIU as released or directly deletes its quota record in its internal global quota management table, logically returning this quota to the available resource pool of the compute node.

[0105] After successfully reclaiming the cache quota, the server can also assess the impact of this resource change on the overall resource utilization of the compute nodes.

[0106] The server determines whether the current cache utilization rate of the compute node is lower than a preset node scaling-down threshold. Specifically, the server can recalculate the total amount of cache actually used by all remaining resource isolation units on the compute node after the reclamation operation, and compare it with the total available cache capacity of the compute node to obtain the node-level cache utilization rate.

[0107] If the server determines that the current cache utilization of a compute node is lower than a preset node scaling-down threshold, it indicates that the node's overall resource load is too low after deleting a resource isolation unit, resulting in wasted resource idleness. In this case, the server will trigger a scaling-down operation on the compute node. The server may request a reduction in the capacity of the persistent storage volume attached to the compute node by calling the cloud platform's application programming interface, thereby reducing infrastructure costs.

[0108] For example, after removing a large data analytics tenant, the node utilization rate drops from 60% to 15%, which is far below the 30% node shrinkage threshold. The server will then automatically initiate and complete the shrinkage of node storage.

[0109] This trigger node scaling down operation can automatically and precisely map the reduction of upper-layer application load to the reduction of underlying storage capacity. This reflects the system's design philosophy of moving from passive management to proactive optimization, and from local resource adjustment to global cost coordination. It is a key link in achieving the goals of efficient resource utilization and precise control of operating costs.

[0110] Furthermore, in the embodiments of this specification, when some data in a resource isolation unit is migrated to other computing nodes, its data scale and access patterns may undergo fundamental changes, and the original cache quota may no longer be reasonable. Therefore, the server sequence accurately identifies the resource redundancy generated after data migration and automatically completes resource reclamation, while also taking into account global optimization at the node level.

[0111] Specifically, this involves the server handling data migration from resource isolation units. When the server receives a migration request from a resource isolation unit, and the request explicitly specifies the amount of data to be migrated, the server coordinates the remote replication and consistency maintenance of the data.

[0112] After the data migration is completed, the total amount of data managed locally by the resource isolation unit is reduced, and the server needs to assess whether this means that its caching resource requirements are also reduced.

[0113] Therefore, after the data migration operation is successfully completed, the server does not immediately adjust the quota. Instead, it initiates a security assessment process. First, it determines whether the current cache utilization of the resource isolation unit is lower than the preset unit scaling-down threshold. This confirms whether the remaining cache resources of the resource isolation unit are sufficient after the data migration to safely reduce the quota without immediately causing resource strain. Simultaneously, the server also determines whether the current quota of the resource isolation unit is not lower than the guaranteed minimum cache quota. This determination ensures that any quota adjustment will not erode the resource guarantees enjoyed by the resource isolation unit under the service level agreement.

[0114] Subsequently, if the server determines that both of the above conditions are true, it indicates that the resource isolation unit does indeed have surplus resources after data migration, and reducing the quota will not touch its guarantee bottom line. At this time, the server will reclaim a portion of the cache quota of the resource isolation unit based on the amount of migrated data. The amount reclaimed can be proportional to the amount of migrated data or calculated according to other strategies.

[0115] After completing the quota reclamation at the unit level, the server can also determine whether the current cache utilization of the compute node is lower than the preset node scaling-down threshold. If the overall resource utilization of the compute node also drops to a low level due to this resource isolation unit quota reclamation, the server will scale down the compute node to save underlying physical resources.

[0116] For example, a content management tenant migrates half of its archived image data to cold storage. After this, its cache utilization drops to a very low level and its quota is far above the lower limit. The server then reclaims a portion of its cache quota proportional to the amount of data migrated. Subsequently, it is found that the total node utilization has also dropped below the scaling-down threshold, which further triggers the scaling-down of the node's storage disk.

[0117] If the server's judgment concludes that either of the above conditions is false, it means that either the resource isolation unit itself does not have sufficient cache resources (i.e., the utilization rate is not lower than the threshold), or its quota is close to or at the guaranteed lower limit. In both cases, to avoid impairing performance or violating service commitments, the server will adopt a conservative strategy, not adjusting the cache quota of the resource isolation unit, and waiting for subsequent load changes or other adjustment opportunities.

[0118] Figure 3This diagram illustrates data migration as provided in the embodiments of this specification. The process begins with the server receiving a migration request carrying the amount of data to be migrated and completing the data migration operation. Subsequently, the server performs a security assessment to determine whether two conditions are met simultaneously: the cache utilization rate of the resource isolation unit is lower than a preset unit scaling-down threshold, and its current quota is not lower than the guaranteed cache quota lower limit. If either condition is not met, the process ends, and the unit quota is not adjusted to ensure its performance stability. If both conditions are met, a portion of the unit's cache quota is reclaimed based on the amount of migrated data. After completing the unit-level reclamation, the server further determines whether the cache utilization rate of the compute node is lower than a preset node scaling-down threshold; if it is lower, a scaling-down operation is triggered on the compute node itself. This process ensures that redundant cache resources can be safely and automatically reclaimed after data migration, and may further contribute to infrastructure cost optimization.

[0119] Similarly, in the embodiments of this specification, when some data of the resource isolation unit is migrated to the computing node, in order to ensure the smooth progress of the data migration operation, while preventing migration failure due to insufficient resources or causing performance impact on existing services, and taking into account the expansion cost control in this process.

[0120] First, the server receives a migration request from the resource isolation unit, which explicitly specifies the amount of data to be migrated. Data migration typically occurs in scenarios such as data redistribution, load balancing, or business integration. The server can assess whether the resource isolation unit has sufficient local cache space to receive the data to be migrated, ensuring that the migrated data can be effectively cached. This avoids situations where insufficient caching prevents new data from residing locally, leading to frequent accesses to remote storage and thus negating the purpose of the migration.

[0121] Therefore, the server can determine whether the current quota of a resource isolation unit (RIU) is sufficient to migrate the required amount of data, based on the remaining cache space of that RIU. The remaining cache space refers to the difference between the current quota of the RIU and the amount of cache already used. The server compares this remaining space with the amount of data to be migrated.

[0122] If the remaining space is greater than or equal to the amount of data to be migrated, the server determines that the conditions are met and the data migration can proceed directly. For example, a tenant that receives historical archived data has a current quota of 200GB, of which 50GB has been used, leaving 150GB remaining. The amount of archived data to be migrated is 100GB. The server determines that its current quota meets the migration requirements and then coordinates and initiates the data migration process.

[0123] If the server determines that the current quota of a resource isolation unit does not meet the amount of data to be migrated, that is, the remaining cache space is less than the amount of data to be migrated, it means that direct migration will cause the cache of that resource isolation unit to overflow.

[0124] At this point, the server will not immediately perform data migration, but will instead determine the overall resource status of the compute nodes. The server will determine whether the current cache utilization of the compute nodes exceeds a preset node expansion threshold. The node expansion threshold is a policy value that indicates that node resources are becoming strained and may require increased supply.

[0125] This judgment distinguishes between two scenarios: one where the overall resources of the compute nodes are sufficient, and the demand can be met through internal adjustments, such as temporarily borrowing idle quotas from other units or waiting for the internal cache of the resource isolation unit to be naturally replaced, without the need for immediate expansion; the other scenario where compute node resources are generally strained, and expansion is the only option.

[0126] If the current cache utilization of a compute node exceeds the preset node expansion threshold, it indicates that the overall load on the compute node is high and the internal resource allocation space is limited. In this case, the server will expand the compute node, for example, by increasing the capacity of its local storage device, to improve the total resource supply of the node. After the compute node expansion operation is completed and confirmed, the server will then perform data migration. For example, when merging two business systems, if the target tenant needs to receive 120GB of data, but its remaining space is only 80GB, and the overall compute node utilization has reached 85%, exceeding the 80% expansion threshold, the server will first perform node expansion before completing the data migration.

[0127] If the current cache utilization of a compute node is not greater than the preset node expansion threshold, it indicates that the overall resources of the compute node are still in a relaxed state. In this case, although the current quota of the resource isolation unit is insufficient, the server may choose not to trigger the expensive node expansion for the time being, but instead record the demand and wait for the quota to be increased later through the cache eviction of the resource isolation unit itself or through dynamic quota limit calculation in its next quota adjustment cycle before performing the migration.

[0128] Alternatively, in some implementations, the server may allow migration first, tolerate excessive cache usage in the short term after migration, and rely on subsequent cache eviction mechanisms to gradually reduce the problem.

[0129] Figure 4This diagram illustrates the data migration process provided in the embodiments of this specification. The process begins with the server receiving a migration request carrying the amount of data to be migrated. The server first assesses whether the remaining cache space of the target resource isolation unit is sufficient to accommodate the amount of data to be migrated. If it is sufficient, the data migration is performed directly. If it is not sufficient, the process enters the deliberate expansion judgment stage: the server assesses whether the current cache utilization rate of the computing nodes has exceeded the preset node expansion threshold. If it has not exceeded the threshold, it indicates that the overall resource pressure on the nodes is not high, and the system may cope through internal adjustments or by relying on subsequent cache eviction mechanisms, so the data migration is performed directly. If the node utilization rate has exceeded the expansion threshold, it indicates that resources are generally tight, and the computing nodes need to be expanded first, and the data migration is performed after the expansion is completed. This process reflects a layered, on-demand resource supply strategy, ensuring the reliability of the migration while avoiding unnecessary premature or excessive expansion.

[0130] The above is a method for managing cache quotas in shared storage provided by the embodiments of this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0131] Figure 5 This specification provides a schematic diagram of an apparatus for managing cache quotas in shared storage, as illustrated in an embodiment of the present specification. The apparatus includes: The receiving module 601 is used to receive cache quota update requests from resource isolation units; The acquisition module 602 is used to acquire the quota calculation parameters of the resource isolation unit. The quota calculation parameters include at least the total amount of persistent data on the shared storage and the read cache ratio. The quota update module 603 is used to redetermine the upper limit of the dynamic cache quota of the resource isolation unit on the computing node based on the quota calculation parameters and the number of replicas corresponding to the resource isolation unit; The quota management module 604 is used to adjust the cache resources allocated to the resource isolation unit according to the quota limit.

[0132] Optionally, the quota update module 603 is used to determine the read cache demand budget based on the total amount of persistent data and the read cache ratio; and to redetermine the quota limit of the dynamic cache of the resource isolation unit on the computing node by allocating the read cache demand budget according to the number of replicas corresponding to the resource isolation unit.

[0133] Optionally, the quota calculation parameters further include a write cache ratio; the quota update module 603 is used to determine the amount of dirty data in the resource isolation unit; determine the upper limit of the data volume of the resource isolation unit based on the sum of the total amount of persistent data and the amount of dirty data; and determine the write cache requirement budget of the resource isolation unit on the computing node based on the write cache ratio of the upper limit of the data volume.

[0134] Optionally, the quota update module 603 is used to determine an allocation coefficient based on the number of replicas; adjust the larger of the read cache demand budget and the write cache demand budget based on the allocation coefficient to obtain the allocated quota upper limit; determine whether the allocated quota upper limit is greater than the cache quota lower limit based on a preset cache quota lower limit; if so, use the allocated quota upper limit as the dynamic cache quota upper limit; if not, use the cache quota lower limit as the dynamic cache quota upper limit.

[0135] Optionally, the quota management module 604 is used to determine the current quota of the resource isolation unit; when the quota limit is greater than the current quota, increase the cache resources allocated to the resource isolation unit based on the quota limit; when the quota limit is not greater than the current quota, adjust the cache resources allocated to the resource isolation unit based on the quota limit.

[0136] Optionally, the quota management module 604 is configured to determine the expected cache resources based on the quota limit and the current quotas of other isolation units in the computing node; determine whether the current cache resources of the node meet the expected cache resources; if so, increase the cache resources allocated to the resource isolation unit based on the quota limit; if not, expand the computing node based on the quota limit, and then increase the cache resources allocated to the resource isolation unit after the expansion is completed.

[0137] Optionally, the quota management module 604 is used to reduce the cache resources allocated to the resource isolation unit if the quota limit is less than the current quota; determine whether the current cache utilization rate of the computing node is lower than a preset node shrinkage threshold; if so, shrink the computing node.

[0138] Optionally, the cache quota update request is generated by the resource isolation unit based on its own cache utilization rate; wherein, when the cache utilization rate is less than a preset unit shrinkage threshold, a shrinkage update request is generated; when the cache utilization rate is greater than a preset unit expansion threshold, an expansion update request is generated. The quota management module 604 is used to increase the cache resources allocated to the resource isolation unit based on the quota limit according to the expansion update request; or, based on the quota limit according to the shrink update request, decrease the cache resources allocated to the resource isolation unit.

[0139] Optionally, the device further includes: a unit deletion module 605; The unit deletion module 605 is used to respond to the deletion instruction of the resource isolation unit, reclaim the current cache quota of the resource isolation unit; determine whether the current cache utilization rate of the computing node is lower than a preset node shrinkage threshold; if so, shrink the computing node.

[0140] Optionally, the device further includes: a migration module 606; The migration module 606 is used to receive a migration request from a resource isolation unit, the migration request carrying the amount of migration data; when data migration is completed based on the migration request, it determines whether the cache utilization rate of the resource isolation unit is lower than a preset unit shrinkage threshold, and whether the current quota of the resource isolation unit is not lower than the lower limit of the cache quota; if both are true, then based on the amount of migration data, a portion of the cache quota of the resource isolation unit is reclaimed, and when the current cache utilization rate of the compute node is lower than a preset node shrinkage threshold, the compute node is shrunk; if either is false, then the cache quota of the resource isolation unit is not adjusted. Optionally, the device further includes: a migration module 607; The migration module 607 is used to receive a migration request from a resource isolation unit, the migration request carrying the amount of migration data; based on the remaining cache space of the resource isolation unit, it determines whether the current quota of the resource isolation unit meets the amount of migration data; if not, when the cache utilization rate of the computing node is greater than a preset node expansion threshold, the computing node is expanded and then the data migration is performed; if yes, the data migration is performed.

[0141] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the method for storing metadata in shared storage as described above.

[0142] based on Figure 1 The method for managing cache quotas in shared storage shown in this specification also provides embodiments that... Figure 6 The diagram shows the structure of the electronic device. Figure 6At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the aforementioned method of shared memory cache quota management.

[0143] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for managing cache quotas in shared storage, the method being applied to a compute node, comprising: Receive cache quota update requests from resource isolation units; Obtain the quota calculation parameters of the resource isolation unit, wherein the quota calculation parameters include at least: the total amount of persistent data on the shared storage and the read cache ratio; Based on the quota calculation parameters and the number of replicas corresponding to the resource isolation unit, the quota limit of the dynamic cache of the resource isolation unit on the computing node is re-determined; The cache resources allocated to the resource isolation unit are adjusted according to the quota limit.

2. The method as described in claim 1, wherein the upper limit of the dynamic cache quota of the resource isolation unit on the computing node is re-determined based on the quota calculation parameters and the number of replicas corresponding to the resource isolation unit, specifically includes: Based on the total amount of persistent data and the read cache ratio, determine the read cache requirement budget based on the total amount of persistent data; Based on the read cache demand budget, the quota is allocated according to the number of replicas corresponding to the resource isolation unit, and the upper limit of the dynamic cache quota of the resource isolation unit on the computing node is re-determined.

3. The method as described in claim 2, wherein the quota calculation parameters further include a write cache ratio; Based on the read cache demand budget, the quota is allocated according to the number of replicas corresponding to the resource isolation unit, and the upper limit of the dynamic cache quota of the resource isolation unit on the compute node is re-determined, specifically including: Determine the amount of dirty data in the resource isolation unit; The upper limit of the data volume of the resource isolation unit is determined based on the sum of the total amount of persistent data and the amount of dirty data. Based on the write cache ratio of the upper limit of the data volume, the write cache requirement budget of the resource isolation unit on the computing node is determined.

4. The method as described in claim 2, wherein the quota limit for the dynamic cache of the resource isolation unit on the compute node is re-determined based on the read cache demand budget and the write cache demand budget, according to the number of replicas corresponding to the resource isolation unit, and the quota is distributed accordingly, specifically including: Determine the allocation coefficient based on the number of copies; Based on the allocation coefficient, the larger of the read cache demand budget and the write cache demand budget is adjusted to obtain the allocated quota limit. Based on the preset lower limit of the cache quota, determine whether the upper limit of the allocated quota is greater than the lower limit of the cache quota; If so, the allocated quota limit shall be used as the dynamic cache quota limit. If not, then the lower limit of the cache quota shall be used as the upper limit of the dynamic cache quota.

5. The method as described in claim 1, wherein adjusting the cache resources allocated to the resource isolation unit according to the quota limit specifically includes: Determine the current quota of the resource isolation unit; When the quota limit is greater than the current quota, increase the cache resources allocated to the resource isolation unit based on the quota limit; When the upper limit of the quota is not greater than the current quota, the cache resources allocated to the resource isolation unit are adjusted based on the upper limit of the quota.

6. The method as described in claim 5, wherein, based on the quota limit, the cache resources allocated to the resource isolation unit are increased, specifically including: Based on the quota limit and the current quota of other isolated units in the compute node, determine the expected cache resources; Determine whether the current cached resources of the node meet the expected cached resources; If so, then based on the quota limit, increase the cache resources allocated to the resource isolation unit; If not, the computing node is expanded based on the quota limit, and after the expansion is completed, cache resources allocated to the resource isolation unit are added.

7. The method as described in claim 5, wherein adjusting the cache resources allocated to the resource isolation unit based on the quota limit specifically includes: If the upper limit of the quota is less than the current quota, reduce the cache resources allocated to the resource isolation unit; Determine whether the current cache utilization rate of the computing node is lower than a preset node shrinkage threshold; If so, the computing node is scaled down.

8. The method as described in claim 1, wherein the cache quota update request is generated by the resource isolation unit based on its own cache utilization rate; in, When the cache utilization rate is less than the preset unit shrinkage threshold, a shrinkage update request is generated; when the cache utilization rate is greater than the preset unit expansion threshold, an expansion update request is generated. Based on the quota limit, the cache resources allocated to the resource isolation unit are adjusted, specifically including: Based on the expansion update request, and according to the quota limit, the cache resources allocated to the resource isolation unit are increased; or, Based on the scaling-down update request, and according to the quota limit, the cache resources allocated to the resource isolation unit are reduced.

9. The method of claim 1, further comprising: In response to a deletion command for a resource isolation unit, the current cache quota of the resource isolation unit is reclaimed; Determine whether the current cache utilization rate of the computing node is lower than a preset node shrinkage threshold; If so, the computing node is scaled down.

10. The method of claim 1, further comprising: Receive a migration request from a resource isolation unit, wherein the migration request carries a migration data amount; When data migration is completed based on the migration request, it is determined whether the cache utilization rate of the resource isolation unit is lower than the preset unit shrinkage threshold, and whether the current quota of the resource isolation unit is not lower than the lower limit of the cache quota. If both are true, then based on the amount of migrated data, a portion of the cache quota of the resource isolation unit is reclaimed, and the compute node is scaled down when the current cache utilization rate of the compute node is lower than the preset node scaling-down threshold. If any of them are not true, the cache quota of the resource isolation unit will not be adjusted.

11. The method of claim 1, further comprising: Receive a migration request from a resource isolation unit, the migration request carrying a migration data amount; Based on the remaining cache space of the resource isolation unit, determine whether the current quota of the resource isolation unit meets the migration data volume; If not, when the cache utilization of the computing node is greater than the preset node expansion threshold, the computing node will be expanded and then data migration will be performed. If so, then perform data migration.

12. An apparatus for managing cache quotas in shared storage, the apparatus comprising: The receiving module is used to receive cache quota update requests from resource isolation units; The acquisition module is used to acquire the quota calculation parameters of the resource isolation unit, wherein the quota calculation parameters include at least: the total amount of persistent data on the shared storage and the read cache ratio; The quota update module is used to redetermine the upper limit of the dynamic cache quota of the resource isolation unit on the computing node based on the quota calculation parameters and the number of replicas corresponding to the resource isolation unit; The quota management module is used to adjust the cache resources allocated to the resource isolation unit according to the quota limit.

13. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1-11.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1-11.

Images