Cache partition resource scheduling method, device, electronic device, medium and product

Abstract

This application discloses a method, device, electronic device, medium and product for scheduling cache partition resources, which relates to the field of computer storage technology. The method includes first obtaining the resource usage of multiple cache partitions to provide a basis for resource adjustment; then determining the partition to be expanded and the target partition based on the information, the former accurately meets the resource demand, and the latter clearly defines the partition where idle resources can be allocated, changing the traditional situation where idle resources are difficult to utilize; then migrating part of the resources of the target partition to the partition to be expanded to achieve cache resource reallocation. Compared with the traditional scheme of isolating reserved resources for each partition, this application supports the dynamic flow of resources across regions, scheduling based on actual usage, matching loads and avoiding restrictions; and achieving dynamic balance and efficient utilization of cache resources through predictive scheduling and early allocation.

Description

Technical Field

[0001] The present application relates to the field of computer storage technology, and in particular to a method, device, electronic device, medium and product for scheduling cache partition resources. Background Art

[0002] In computer storage systems, cache modules are typically divided into multiple partitions for data management and performance optimization. Each partition is allocated a fixed amount of resources during system initialization to ensure cached data can be recovered after a power outage. Resource allocation is done on a page-by-page basis. Due to differences in the performance of the backend disk arrays corresponding to different partitions and the varying workloads on the frontend of each partition, resource imbalances are common across partitions.

[0003] The technical solution to resource imbalance is primarily implemented by setting reserved space for each partition. For example, when the total available system resource count is 100 and there are three partitions, each partition is initially reserved with 40 resources, allowing partitions with insufficient backend performance or high frontend pressure to use more resources. However, while this solution can meet the resource needs of some high-demand partitions, it cannot effectively utilize the idle resources of low-demand partitions, resulting in resource waste and reduced overall system resource utilization. It also makes it difficult to achieve dynamic balancing and efficient allocation of cache resources, and cannot fully meet the storage system's needs for resource optimization and management. Summary of the Invention

[0004] The present application provides a method, device, electronic device, medium and product for scheduling cache partition resources, so as to at least solve the problem of low overall system resource utilization in related technologies.

[0005] The present application provides a method for scheduling cache partition resources, including: obtaining resource usage of multiple cache partitions; determining a partition to be expanded and a target partition based on the resource usage of the multiple cache partitions; the partition to be expanded is a cache partition with resource requirements, and the target partition is a cache partition that supports resource migration; and determining to migrate at least a portion of the resources of the target partition to the partition to be expanded.

[0006] The present application also provides a scheduling device for cache partition resources, including:

[0007] The acquisition module is used to obtain the resource usage of multiple cache partitions;

[0008] A type definition module is used to determine the partition to be expanded and the target partition based on the resource usage of multiple cache partitions; the partition to be expanded is the cache partition with resource demand, and the target partition is the cache partition that supports resource migration;

[0009] The pre-migration module is used to determine to migrate at least part of the resources of the target partition to the partition to be expanded.

[0010] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned cache partition resource scheduling methods when executing the computer program.

[0011] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned cache partition resource scheduling methods are implemented.

[0012] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned cache partition resource scheduling methods when executed by a processor.

[0013] This application first obtains the resource usage of multiple cache partitions, understands the actual resource usage status of each partition in real time, and provides an accurate basis for subsequent resource adjustments; based on the obtained resource usage, the partition to be expanded and the target partition are determined. The determination of the partition to be expanded can accurately meet the partitions with resource needs, avoiding the problem of insufficient resources caused by fixed reservations in traditional solutions; at the same time, it is clear that the target partition is a partition that supports resource migration, and it is possible to find partitions with idle resources that can be used for allocation, which changes the situation in which idle resources cannot be effectively utilized in traditional solutions; at least part of the resources of the target partition are migrated to the partition to be expanded, and through this dynamic resource migration method, cache resources are redistributed. Therefore, compared with the isolation of reserved resources for each partition in the related art, this application supports the dynamic flow of resources across regions, uses the resource usage of each partition as the scheduling basis, matches resource allocation with actual load, and avoids resource limitations; allocates resources in advance through predictive scheduling, rather than waiting for insufficient resources and then passively adjusting, to achieve dynamic balance and efficient utilization of cache resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 A flow logic diagram for storing data;

[0016] Figure 2 A schematic diagram of a flow chart of a method for scheduling cache partition resources provided in an embodiment of the present application;

[0017] Figure 3 A schematic diagram illustrating resource usage of multiple cache partitions provided in an embodiment of the present application;

[0018] Figure 4 A schematic diagram of resource usage in a zone to be expanded according to an embodiment of the present application;

[0019] Figure 5 A flowchart of another method for scheduling cache partition resources provided in an embodiment of the present application;

[0020] Figure 6 A schematic diagram of the structure of a cache partition resource scheduling device provided in an embodiment of the present application;

[0021] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0024] In order to more clearly illustrate the embodiments of the present application, the following briefly introduces the technical terms used in the embodiments:

[0025] The cache module is a component in a computer system used to temporarily store data. Its fast access feature reduces direct access to slow back-end storage (such as disks) and improves data reading efficiency.

[0026] Cache partitioning divides the cache module into multiple independent areas, each of which can manage its own data caching strategy. Each cache module partition is allocated a fixed amount of resources during system initialization. These resources are used to ensure that cached data can be recovered after a power outage. Resource allocation is done on a page-by-page basis.

[0027] A disk array is a storage unit composed of multiple physical disks, either through hardware or software. It uses technologies like data redundancy and striping to improve storage performance, reliability, or capacity. Each cache partition corresponds to a backend disk array.

[0028] like Figure 1 As shown, Figure 1 This diagram illustrates the logical flow of stored data. Data read / write (IO) requests initiated by the host first reach the cache module. The cache module then distributes these requests to different cache partitions based on specific rules, ensuring that different IOs do not interfere with each other. After processing in the cache partitions, the data is persisted to the corresponding disk array to prevent data loss.

[0029] Solid State Drive (SSD) uses solid-state electronic storage chips (such as flash memory chips) to store data. It has no mechanical moving parts and has the characteristics of fast read and write speeds, good shock resistance, low power consumption, and no noise. It can effectively improve the data processing efficiency of the device.

[0030] A mechanical hard disk drive (HDD) uses rotating magnetic platters and moving heads to read and write data. It has internal mechanical components such as platters, heads, and motors. It has large capacity and low cost, making it suitable for storing large amounts of data.

[0031] A hybrid hard drive (HHD) combines the magnetic storage of a mechanical hard drive with the high-speed cache of flash memory, integrating the advantages of the large capacity and low cost of a mechanical hard drive with the accelerated reading and writing of commonly used data by flash memory.

[0032] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0033] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the cache partition resource scheduling method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0034] The core environmental features of the application environment architecture include: distributed cache cluster architecture, layered cache architecture and containerized / cloud native environment adaptation. Among them, the cache system consists of a cluster of multiple physical nodes or virtual nodes, and each node carries several cache partitions. Partition mapping relationships, resource quotas and status information are maintained through a central controller or distributed coordination service. High-speed network connections are required between nodes to support real-time migration of resources. The scheduling method needs to perceive the resource pressure of each layer of cache, combine data access frequency analysis, and prioritize the deployment of hot data partitions in the high-performance cache layer, and migrate cold data partitions to the capacity storage layer. Based on the container orchestration platform, cache partitions can be mapped to container group resources and dynamically adjust resource quotas.

[0035] The hardware architecture includes computing and storage hardware, hardware acceleration components, and hardware monitoring modules. Computing and storage hardware includes central processing units (CPUs), memory, storage media, and network devices. Hardware acceleration components include accelerators and smart network cards (SNICs). Accelerators are used to offload compute-intensive tasks such as cache data compression and encryption, freeing up CPU (Central Processing Unit) resources for scheduling logic. Smart network cards support direct writing of network data into memory, reducing CPU interrupt overhead during resource migration. The hardware monitoring module includes resource monitoring sensors and performance counters. Temperature, voltage, and memory usage sensors integrated into the server motherboard provide real-time feedback on hardware health status. Performance counters record metrics such as cache hit rate and memory bandwidth utilization, providing data support for scheduling decisions.

[0036] An embodiment of the present application provides a method for scheduling cache partition resources. The method is described in detail in conjunction with the execution flow of the method for scheduling cache partition resources.

[0037] like Figure 2 As shown, Figure 2 A schematic flow chart of a method for scheduling cache partition resources provided in an embodiment of the present application, the method comprising the following steps S201 to S203.

[0038] S201: Obtain resource usage of multiple cache partitions.

[0039] Multiple cache partitions are obtained by dividing the cache module according to specific rules. Resource usage includes at least one of resource usage rate, whether there are idle resources, the current maximum number of available resources, and the number of read and write operations to be allocated. Among them, the read and write operations to be allocated are read and write operations waiting for resources. Figure 3 The figure shows the resource usage of multiple cache partitions, including the partition's initial maximum available resources, the partition's target resources, the partition's current maximum available resources, the partition's idle resources, and the partition's used resources. The partition's initial maximum available resources represent the maximum resources initially allocated to the partition; the partition's target resources represent the resources the partition ultimately expects to reach; the partition's current maximum available resources represent the maximum resources the partition can actually call; the partition's idle resources represent idle resources; and the partition's used resources represent occupied resources. The phrase "Partition's current maximum available resources = Partition's idle resources + Partition's used resources" in the figure indicates that the partition's used resources can be released, making resource allocation more flexible and available.

[0040] In some embodiments, before step S201, the method further includes: obtaining a maximum total number of available resources for the plurality of cache partitions, and allocating an initial maximum number of available resources to the plurality of cache partitions based on the maximum total number of available resources and the number of cache partitions. The maximum total number of available resources is the maximum total number of resources that can be used by the cache module during system initialization.

[0041] It can be understood that the maximum total available resources of the cache module are obtained, and then the initial value of the maximum available resources of each cache partition is calculated based on the maximum total available resources and the number of cache partitions, thereby initializing the maximum available resources of each cache partition. This evenly distributes the total resources to each partition and avoids resource allocation imbalance.

[0042] Optionally, the process of determining the initial maximum number of available resources for each cache partition based on the maximum total number of available resources and the number of multiple cache partitions includes: first calculating the quotient and remainder of the maximum total number of available resources and the number of multiple cache partitions. Then, the sum of the quotient and the remainder is used as the initial maximum number of available resources for the first cache partition among the multiple cache partitions, and the quotient is used as the initial maximum number of available resources for all cache partitions other than the first cache partition. The quotient is the default number of basic resources allocated to each cache partition, and the remainder is the number of resources remaining after the basic resources are allocated.

[0043] As can be understood, the maximum available total is divided by the number of partitions to obtain a quotient or a quotient and a remainder. If there is a remainder, the sum of the quotient and the remainder is used as the initial maximum available resource count for the first cache partition. The initial maximum available resource counts for the remaining cache partitions are equal to the quotient. If there is no remainder, the initial maximum available resource count for each cache partition is the same and equal to the quotient.

[0044] As an optional implementation provided in an embodiment of the present application, after calculating the quotient and remainder of the maximum total available resources and the number of cache partitions, the remainder can be evenly distributed among some of the cache partitions. For example, assuming there are N cache partitions, the quotient of the maximum total available resources and N is represented by Q, and the remainder is represented by R. Q resources are allocated to NR cache partitions, and the remaining R resources are evenly distributed among the R partitions.

[0045] The above embodiment ensures that there is no surplus in the initial allocation of total resources, fully utilizes all available resources, and realizes rapid and balanced allocation of resources during system initialization.

[0046] In some embodiments, resource usage of multiple cache partitions is periodically counted by a timer. The resource usage of multiple cache partitions includes any of the following: (1) a probability value of satisfying a preset condition within a historical period is greater than a preset probability value, wherein the preset condition includes a resource usage rate greater than a preset usage rate and a first number of read and write operations to be allocated being greater than or equal to a first threshold; (2) a current maximum number of available resources is greater than or equal to a preset number of resources and there are idle resources; (3) a current maximum number of available resources is greater than or equal to a preset number of resources, there are no idle resources, but a second number of read and write operations to be allocated is less than or equal to a second threshold; (4) a current maximum number of available resources is less than a preset number of resources.

[0047] The above implementation continuously monitors resource usage and facilitates real-time adjustment of resource allocation.

[0048] S202: Determine a partition to be expanded and a target partition based on resource usage of multiple cache partitions.

[0049] The partition to be expanded is a cache partition with resource requirements and is the partition that needs to have its resources expanded. The target partition is a cache partition that supports resource migration, indicating that its resources can be loaned to other cache partitions. The partition to be expanded and the target partition are two types of cache partitions among multiple cache partitions. This application also includes other types of cache partitions, such as resource-private partitions. A resource-private partition indicates that the resources of this cache partition cannot be loaned to other cache partitions.

[0050] For ease of description, the process of determining the cache partition type will be described below using any different cache partition among the multiple cache partitions, such as a first cache partition and a second cache partition.

[0051] In some embodiments, the partition to be expanded is determined based on the resource usage of multiple cache partitions, including: calculating the probability value of the first cache partition meeting preset conditions within a historical period, wherein the preset conditions include that the resource usage rate is greater than the preset usage rate and the first number of read and write operations to be allocated is greater than or equal to a first threshold; when the probability value is greater than the preset probability value, determining that the first cache partition is the partition to be expanded.

[0052] The resource utilization rate in the preset condition is greater than the preset utilization rate, indicating that the core resources of the first cache partition are occupied and the idle resources are limited. An exemplary preset utilization rate can be 100%. The first number of read and write operations to be allocated is greater than or equal to the first threshold, indicating that when the resource utilization rate is saturated, new read and write operations cannot be processed immediately and will enter the waiting queue. If the number of read and write operations waiting to be processed exceeds the first threshold, it means that the resource bottleneck has caused a backlog of read and write blocks. The probability value of meeting the preset conditions in the historical period is greater than the preset probability value, indicating that the above situation frequently occurs in the historical period, indicating that insufficient resources have become the norm. Based on this, it can be determined that the first cache partition is a cache partition with resource needs, that is, a partition to be expanded.

[0053] Specifically, for the first cache partition, a determination is first made as to whether its resource utilization rate during a historical period is greater than a preset utilization rate, and whether a first number of pending read and write operations is greater than or equal to a first threshold. If so, indicating resource saturation and an excess of pending read and write operations, the probability of these conditions occurring during the historical period is calculated to determine whether it is greater than a preset probability value. If the probability of these conditions occurring during the historical period is greater than the preset probability value, excluding sporadic occurrences, the first cache partition is determined to be a partition to be expanded.

[0054] For example, Figure 4 The resource count for the partition to be expanded, i.e., the partition to be expanded, includes the initial maximum available resources, the target resources, the current maximum available resources, the idle resources, and the used resources. "Idle resources = 0" indicates that the partition to be expanded currently has no resource limits and all resources are occupied. "Used resources = Current maximum available resources" indicates that the partition has no idle resources and the current upper limit is equal to the used resources.

[0055] The above embodiment determines that the cache partition is the partition to be expanded by the situation where the cache partition resources are saturated and continuously affecting the business, thereby avoiding blind expansion; normalization problems are judged by historical probability, avoiding system crashes due to long-term resource shortages, and realizing accurate positioning of resource bottlenecks.

[0056] In some embodiments, determining a target partition based on resource usage across multiple cache partitions includes: obtaining the current maximum available resource count of a second cache partition; if the current maximum available resource count is greater than or equal to a preset resource count, determining whether the second cache partition has idle resources; and if the second cache partition has idle resources, determining the second cache partition as the target partition. Optionally, the preset resource count is calculated based on a preset coefficient and an initial maximum available resource count, for example, the preset resource count is 10% of the initial maximum available resource count.

[0057] Specifically, a determination is made as to whether the second cache partition's current maximum available resources are greater than or equal to a preset number of resources. If so, this ensures that basic available resources remain after the second cache partition's resources are loaned out. A further determination is made as to whether the second cache partition has idle resources. The presence of idle resources in the second cache partition indicates that the second storage partition has available resources for loaning out, which is a prerequisite for the second cache partition to loan out resources. The second cache partition is determined to support resource migration, indicating that it is the target partition.

[0058] The above embodiment combines a pre-set resource count with a double check of idle resources to ensure that the second cache partition still has sufficient idle resources after borrowing resources, thus preventing the second cache partition from running out of resources due to borrowed resources. This method of identification identifies target partitions with idle resources that can be migrated and loaned to other cache partitions, facilitating dynamic resource scheduling across cache partitions and improving resource utilization. Compared to traditional technologies, dynamic resource migration can meet the fluctuating resource demands of multiple partitions.

[0059] In other embodiments, determining the target partition based on the resource usage of multiple cache partitions also includes: if there are no idle resources in the second cache partition, determining whether the second number of read and write operations to be allocated in the second cache partition within the historical period is less than or equal to a second threshold; if the second number is less than or equal to the second threshold, determining the second cache partition as the target partition.

[0060] Specifically, the second cache partition's current maximum available resources are greater than or equal to the preset number of resources, but it has no idle resources, indicating that the second cache partition's resources are fully occupied. However, its second number of pending read and write operations within the historical period is less than or equal to the second threshold, indicating that resource occupation has not caused read and write blocking. In this resource usage scenario, although the second cache partition has no idle resources, it does not cause read and write blocking due to insufficient resources. This indicates that the second cache partition has high actual processing efficiency, and borrowing some resources is safe and will not affect its performance, making it a suitable target partition.

[0061] The above embodiment is applicable to resource-intensive but low-latency business scenarios. Through historical data verification, it can be determined that a resource-saturated but efficient cache partition is used as the target partition, which helps to avoid resource waste and ensure the security of resource migration.

[0062] In some embodiments, a resource-private partition can also be determined based on the resource usage of multiple cache partitions, including: obtaining the current maximum number of available resources of the second cache partition, and when the current maximum number of available resources of the second cache partition is less than or equal to the preset number of resources, determining that the second cache partition is a resource-private partition, indicating that the resources of the second cache partition cannot be migrated to any other cache partition.

[0063] In the above embodiment, if the current maximum available resource count of a cache partition is less than or equal to the preset resource count, this indicates that the cache partition is resource-constrained. Borrowing resources will lead to resource exhaustion, making it unable to cope with potential sudden requests and business fluctuations. Therefore, it is marked as a resource-private partition to ensure the basic availability of the cache partition and to some extent help balance global and local resource allocation.

[0064] S203: Determine to migrate at least a portion of the resources of the target partition to the partition to be expanded.

[0065] If the target partitions are different, at least a portion of the resources migrated to the partition to be expanded will be different.

[0066] In some embodiments, if the target partition has idle resources, it is determined that the idle resources will be migrated to the partition to be expanded. If the target partition has idle resources, the idle resources can be migrated to the partition to be expanded that has resource demand, thereby avoiding resource fragmentation and waste, achieving dynamic resource scheduling, and improving overall utilization.

[0067] Based on the above embodiment, if there are idle resources in the target partition, the number of idle resources is determined based on these idle resources, and then the current maximum available resource number is updated based on the current maximum available resource number and the number of idle resources of the target partition.

[0068] Specifically, the new current maximum available resource count = the current maximum available resource count - the number of idle resources. By modifying the available resource cap for the target cache partition, the system is encouraged to allocate idle resources to the partition to be expanded. This also limits resource requests from subsequent pending read and write operations, avoiding excessive resource waste or restrictions and improving global resource utilization.

[0069] In some embodiments, if the target partition has no idle resources, a target number of available resources is determined based on a preset coefficient and the current maximum number of available resources. Then, based on the current maximum number of available resources and the target number of available resources, the number of resources to be released is determined. Furthermore, it is determined that the resources to be released corresponding to the number of resources to be released are migrated to the partition to be expanded. The preset coefficient may be 90%.

[0070] Specifically, if there are no idle resources in the target partition, it means that the resources of the target partition are being occupied by read and write operations. The product of the preset coefficient and the current maximum number of available resources is calculated to obtain the target number of available resources. The difference between the current maximum number of available resources and the target number of available resources is calculated to obtain the number of resources to be released. After the read and write operations are completed, the resources will be released. In this application, the resources will not be directly returned to the target partition, but it will be determined to migrate these resources to be released to the partition to be expanded to avoid the situation where the resources are physically occupied but logically idle. Waiting for the resources to be released to be migrated to the partition to be expanded to avoid interrupting the read and write operations being processed,

[0071] The above embodiment migrates resources from the target partition to the partition to be expanded without pre-allocating fixed reserved space. Through the dynamic migration mechanism, the system can flexibly adjust resources according to actual needs without manual intervention, thereby achieving dynamic balance of cache resources, improving the overall resource utilization of the system, and better meeting the storage system's needs for resource optimization management.

[0072] Based on the above embodiment, the current maximum available resource count of the target partition is updated based on the target available resource count. Optionally, during the update of the current maximum available resource count of the target partition, as the number of resources released by the pending read and write operations is allocated, the current maximum available resource count is gradually deducted until it equals the target available resource count. The current maximum available resource count is set to the target resource count. After migrating the pending released resources, the target partition still has sufficient resources to handle read and write operations. This not only meets the needs of other partitions but also ensures the stability of the target partition itself.

[0073] The above embodiment dynamically adjusts the upper limit of available partition resources, achieving efficient scheduling and balancing of global resources while ensuring the IO processing capability of a single cache partition, avoiding idle resources or imbalanced allocation, and improving the overall performance and stability of the storage system.

[0074] In summary, this application first obtains the resource usage of multiple cache partitions. Unlike the fixed reserved space in traditional solutions, this method provides real-time information on the actual resource usage of each partition, providing an accurate basis for subsequent resource adjustments and helping to identify idle resources in low-demand partitions and resource shortages in high-demand partitions. Based on the obtained resource usage, the partition to be expanded and the target partition are determined. The determination of the partition to be expanded can accurately meet the partition with resource demand, avoiding the resource shortage problem caused by fixed reservations in traditional solutions. At the same time, by clearly defining the target partition as a partition that supports resource migration, it is possible to find partitions with idle resources that can be allocated, changing the situation in which idle resources cannot be effectively utilized in traditional solutions. At least a portion of the resources of the target partition are migrated to the partition to be expanded. Through this dynamic resource migration method, cache resources are redistributed. Compared with the fixed resource reservation mode in traditional solutions, this method can flexibly adjust resources according to actual needs, allowing resources to flow from low-utilization partitions to high-utilization partitions, thereby achieving dynamic balancing of cache resources, improving overall system resource utilization, and better meeting the storage system's needs for resource optimization and management. The cache partition resource scheduling method achieves balanced resource allocation through dynamic resource migration, effectively solving the problems of traditional reserved space solutions.

[0075] The embodiment of the present application provides another method for scheduling cache partition resources, such as Figure 5 As shown, the method includes the following steps S501 to S505:

[0076] S501: Obtain resource usage of multiple cache partitions.

[0077] S502: Determine a partition to be expanded and a target partition according to resource usage of multiple cache partitions.

[0078] The specific implementation of the above steps can refer to the aforementioned steps S201~S202, and this application will not go into details here.

[0079] S503: When it is determined based on resource usage of multiple cache partitions that there is a partition to be expanded but there is no target partition, data required for the read and write operations to be allocated for the partition to be expanded is loaded into the partition to be expanded.

[0080] The read and write operations to be allocated for the partition to be expanded are triggered in response to a read and write request submitted by the host.

[0081] S504: Synchronize the data required for the read and write operations to be allocated and stored in the partition to be expanded to the disk array.

[0082] S505: After receiving the response returned by the disk array, a response is returned to the host indicating that the read and write operations to be allocated are completed.

[0083] When it is determined based on the resource usage of multiple cache partitions that there is a partition to be expanded, but there is no target partition, it means that the resource usage of all cache partitions is almost saturated, and the global resource tension problem cannot be solved through internal resource scheduling. To address this problem, the present application first loads the data required for the read and write operations to be allocated for the partition to be expanded into the partition to be expanded, and then flushes these data to the disk array, waiting for the disk array to respond. After the disk array responds, a response is sent to the host indicating that the read and write operations to be allocated are completed. Compared with the traditional read and write operation processing flow in which "host response time = partition processing time", the present application waits for the disk array to respond before replying to the host, so that "host response time = partition processing time + disk response time", lengthens the time of the IO processing link, and makes the host perceive that the IO processing has slowed down, thereby reducing the number of IOs submitted per unit time.

[0084] The above embodiment can reduce the rate at which the host issues IO by extending the processing time of read and write operations, thereby realizing the interaction between storage and host, alleviating the pressure on storage resources, and avoiding cache overload.

[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0086] like Figure 6As shown, an embodiment of the present application further provides a cache partition resource scheduling device, the device comprising:

[0087] Acquisition module 601, used to obtain resource usage of multiple cache partitions;

[0088] Type definition module 602, configured to determine a partition to be expanded and a target partition based on resource usage of multiple cache partitions; the partition to be expanded is a cache partition with resource requirements, and the target partition is a cache partition that supports resource migration;

[0089] The pre-migration module 603 is configured to determine to migrate at least a portion of the resources of the target partition to the partition to be expanded.

[0090] As an optional implementation of an embodiment of the present application, the type definition module 602 is specifically used to: calculate the probability value of the first cache partition meeting the preset conditions within a historical period; wherein the first cache partition is any cache partition among multiple cache partitions, and the preset conditions include that the resource utilization rate is greater than the preset utilization rate and the first number of read and write operations to be allocated is greater than or equal to a first threshold; when the probability value is greater than the preset probability value, determine that the first cache partition is the partition to be expanded.

[0091] As an optional implementation manner of an embodiment of the present application, the type definition module 602 is specifically used to: obtain the current maximum number of available resources of the second cache partition; the second cache partition is any cache partition among multiple cache partitions; when the current maximum number of available resources is greater than or equal to the preset number of resources, determine whether there are free resources in the second cache partition; if there are free resources in the second cache partition, determine that the second cache partition is the target partition; if there are no free resources in the second cache partition, determine whether the second number of read and write operations to be allocated to the second cache partition in the historical period is less than or equal to a second threshold; when the second number is less than or equal to the second threshold, determine that the second cache partition is the target partition.

[0092] As an optional implementation of the embodiment of the present application, the pre-migration module 603 is specifically configured to: if there are idle resources in the target partition, determine to migrate the idle resources to the partition to be expanded.

[0093] As an optional implementation of the embodiment of the present application, the device also includes an update module, which is used to: if there are idle resources in the target partition, determine the number of idle resources based on the idle resources in the target partition; and update the current maximum number of available resources based on the current maximum number of available resources and the number of idle resources of the target partition.

[0094] As an optional implementation method of an embodiment of the present application, the pre-migration module 603 is specifically used to: if there are no idle resources in the target partition, determine the target number of available resources based on the preset coefficient and the current maximum number of available resources; determine the number of resources to be released based on the current maximum number of available resources and the target number of available resources; and determine to migrate the resources to be released corresponding to the number of resources to be released to the partition to be expanded.

[0095] As an optional implementation of the embodiment of the present application, the device also includes an updating module, which is used to: update the current maximum number of available resources according to the target number of available resources.

[0096] As an optional implementation of the embodiment of the present application, the device also includes a processing module, which is used to: when it is determined that there is a partition to be expanded based on the resource usage of multiple cache partitions, but there is no target partition, load the data required for the read and write operations to be allocated of the partition to be expanded into the partition to be expanded; synchronize the data required for the read and write operations to be allocated stored in the partition to be expanded to the disk array; and after receiving a response returned by the disk array, return a response to the host to complete the read and write operations to be allocated.

[0097] As an optional implementation of an embodiment of the present application, the type definition module 602 is also used to: when the current maximum number of available resources is less than the preset number of resources, determine that the second cache partition is a resource private partition, indicating that the resources of the second cache partition cannot be migrated to any other cache partition.

[0098] As an optional implementation of the embodiment of the present application, the device also includes an initialization module for: obtaining the maximum total number of available resources of multiple cache partitions; and allocating an initial maximum number of available resources to multiple cache partitions based on the maximum total number of available resources and the number of multiple cache partitions.

[0099] As an optional implementation manner of an embodiment of the present application, the initialization module is specifically used to: calculate the quotient and remainder of the total maximum available resources and the number of multiple cache partitions; use the sum of the quotient and the remainder as the initial maximum available resource number of the first cache partition among the multiple cache partitions, and use the quotient as the initial maximum available resource number of other cache partitions except the first cache partition.

[0100] For the description of the features in the embodiment corresponding to the cache partition resource scheduling device, please refer to the relevant description of the embodiment corresponding to the cache partition resource scheduling method, which will not be repeated here.

[0101] The embodiment of the present application also provides an electronic device, such as Figure 7 As shown, it includes a memory 701 and a processor 702. The memory 701 stores a computer program, and the processor 702 is configured to run the computer program to execute the steps in any of the above-mentioned cache partition resource scheduling method embodiments.

[0102] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned cache partition resource scheduling method embodiments when running.

[0103] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0104] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned cache partition resource scheduling method embodiments are implemented.

[0105] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned cache partition resource scheduling method embodiments.

[0106] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0107] The above is a detailed introduction to the scheduling method, device, electronic device, medium and product for cache partition resources provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for scheduling cache partition resources, characterized in that: include: Get resource usage of multiple cache partitions; Determining a partition to be expanded and a target partition according to resource usage of the multiple cache partitions; The partition to be expanded is a cache partition with resource demand, and the partition to be expanded meets the following conditions: a resource usage rate in a historical period is greater than a preset usage rate, a first number of read and write operations to be allocated is greater than or equal to a first threshold, and a probability of occurrence of this situation is greater than a preset probability value; The target partition is a cache partition that supports resource migration, and the target partition satisfies that the current maximum number of available resources is greater than or equal to a preset number of resources, and there are idle resources or the second number of read and write operations to be allocated in a historical period is less than or equal to a second threshold; Determine to migrate at least a portion of resources of the target partition to the partition to be expanded, where the at least a portion of resources includes idle resources or resources to be released; Determining a target partition based on resource usage of multiple cache partitions includes: obtaining the current maximum number of available resources of the second cache partition; the second cache partition is any cache partition among the multiple cache partitions; when the current maximum number of available resources is greater than or equal to a preset number of resources, determining whether the second cache partition has idle resources; if the second cache partition has idle resources, determining the second cache partition as the target partition; if the second cache partition does not have idle resources, determining whether a second number of read and write operations to be allocated to the second cache partition within a historical period is less than or equal to a second threshold; and determining the second cache partition as the target partition if the second number is less than or equal to the second threshold.

2. The method according to claim 1, characterized in that The determining to migrate at least a portion of the resources of the target partition to the partition to be expanded includes: If there are idle resources in the target partition, it is determined to migrate the idle resources to the partition to be expanded.

3. The method according to claim 1, characterized in that The method further comprises: If there are idle resources in the target partition, determining the number of idle resources according to the idle resources in the target partition; The current maximum number of available resources is updated according to the current maximum number of available resources of the target partition and the number of idle resources.

4. The method according to claim 1, wherein The determining to migrate at least a portion of the resources of the target partition to the partition to be expanded includes: If there are no idle resources in the target partition, determining the target available resource number according to a preset coefficient and the current maximum available resource number; Determining the number of resources to be released according to the current maximum number of available resources and the target number of available resources; Determine to migrate the to-be-released resources corresponding to the number of to-be-released resources to the partition to be expanded.

5. The method according to claim 4, characterized in that The method further comprises: Update the current maximum number of available resources based on the target number of available resources.

6. The method according to claim 1, characterized in that The method further comprises: If it is determined according to resource usage of the multiple cache partitions that there is a partition to be expanded but there is no target partition, then the data required for the read and write operations to be allocated of the partition to be expanded is loaded into the partition to be expanded; Synchronizing the data required for the read and write operations to be allocated and stored in the partition to be expanded to the disk array; After receiving the response returned by the disk array, a response is returned to the host indicating that the read and write operations to be allocated are completed.

7. The method according to claim 1, characterized in that The method further comprises: When the current maximum number of available resources is less than the preset number of resources, the second cache partition is determined to be a resource-private partition, which means that resources of the second cache partition cannot be migrated to any other cache partition.

8. The method according to claim 1, characterized in that Before obtaining resource usage of the plurality of cache partitions, the method further includes: Obtaining the maximum total available resources of the multiple cache partitions; An initial maximum number of available resources is allocated to the plurality of cache partitions according to the maximum total number of available resources and the number of the plurality of cache partitions.

9. The method according to claim 8, characterized in that The allocating an initial maximum number of available resources to the plurality of cache partitions according to the maximum total number of available resources and the number of the plurality of cache partitions includes: Calculating a quotient and a remainder of the maximum total available resources and the number of the plurality of cache partitions; The sum of the quotient and the remainder is used as the initial maximum available resource number of a first cache partition among the multiple cache partitions, and the quotient is used as the initial maximum available resource number of other cache partitions except the first cache partition.

10. A cache partition resource scheduling device, characterized in that: include: The acquisition module is used to obtain the resource usage of multiple cache partitions; A type definition module, configured to determine a partition to be expanded and a target partition according to resource usage of the plurality of cache partitions; The partition to be expanded is a cache partition with resource demand, and the partition to be expanded meets the following conditions: a resource usage rate in a historical period is greater than a preset usage rate, a first number of read and write operations to be allocated is greater than or equal to a first threshold, and a probability of occurrence of this situation is greater than a preset probability value; The target partition is a cache partition that supports resource migration, and the target partition satisfies that the current maximum number of available resources is greater than or equal to a preset number of resources, and there are idle resources or the second number of read and write operations to be allocated in a historical period is less than or equal to a second threshold; A pre-migration module, configured to determine to migrate at least a portion of the resources of the target partition to the partition to be expanded, wherein the at least a portion of the resources includes idle resources or resources to be released; The type definition module is specifically used to: obtain the current maximum available resource number of the second cache partition in the process of determining the target partition according to the resource usage of the multiple cache partitions; The second cache partition is any cache partition among the multiple cache partitions; when the current maximum number of available resources is greater than or equal to the preset number of resources, determining whether there are idle resources in the second cache partition; If there are idle resources in the second cache partition, determining the second cache partition as the target partition; If there are no idle resources in the second cache partition, determine whether the second number of read and write operations to be allocated to the second cache partition in the historical period is less than or equal to a second threshold; if the second number is less than or equal to the second threshold, determine the second cache partition as the target partition.

11. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the cache partition resource scheduling method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method for scheduling cache partition resources according to any one of claims 1 to 9 are implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the cache partition resource scheduling method according to any one of claims 1 to 9 are implemented.

Images