Cache partition resource scheduling method and device, electronic equipment, medium and product

Abstract

The invention discloses a cache partition resource scheduling method and device, electronic equipment, a medium and a product, and relates to the technical field of computer storage, and the method comprises the following steps: firstly obtaining a multi-cache partition resource use condition, and providing a basis for resource adjustment; the to-be-expanded partition accurately meets the resource requirements, and the target partition determines the partition capable of deploying the idle resources, so that the traditional situation that the idle resources are difficult to use is changed; and then migrating part of resources of the target partition to the partition to be expanded to realize cache resource redistribution. Compared with a traditional scheme that reserved resources of all partitions are isolated, the method supports cross-region dynamic circulation of the resources, loads are scheduled and matched according to actual use conditions, and limitation is avoided; and dynamic balance and efficient utilization of cache resources are realized through pre-allocation of predictive scheduling.

Description

Technical Field

[0001] This application relates to the technical field of computer storage, and in particular, to a method, device, electronic device, medium, and product for scheduling cache partition resources. Background Art

[0002] In a computer storage system, a cache module is usually divided into multiple partitions to achieve data management and performance optimization. Each partition is allocated a fixed amount of resources during system initialization to ensure that cached data can be restored after a power failure, and resource requests are made in units of pages. Since the performance of the backend disk arrays corresponding to different partitions varies, and the front-end service pressures of each partition are also different, there is generally an uneven resource usage problem among partitions.

[0003] Related technical solutions to address resource imbalance mainly achieve this by setting reserved spaces for each partition. For example, when the total available resource count of the system is 100 and there are 3 partitions, each partition reserves 40 resources during initialization, allowing partitions with insufficient backend performance or high front-end pressure to use more resources. However, although this solution can meet the resource calls of some high-demand partitions, it cannot effectively utilize the idle resources of low-demand partitions, resulting in resource waste, reducing the overall resource utilization rate of the system, making it difficult to achieve dynamic balance and efficient allocation of cache resources, and unable to fully meet the requirements of the storage system for optimized resource management. Summary of the Invention

[0004] This application provides a method, device, electronic device, medium, and product for scheduling cache partition resources to at least solve the problem of low overall resource utilization rate in related technologies.

[0005] This application provides a method for scheduling cache partition resources, including: obtaining the resource usage conditions of multiple cache partitions; determining an expansion-needed partition and a target partition according to the resource usage conditions of the multiple cache partitions; the expansion-needed partition is a cache partition with resource requirements, and the target partition is a cache partition that supports resource migration; determining to migrate at least a part of the resources of the target partition to the expansion-needed partition.

[0006] This application also provides a device for scheduling cache partition resources, including: An obtaining module, configured to obtain the resource usage conditions of multiple cache partitions; A type-defining module, configured to determine an expansion-needed partition and a target partition according to the resource usage conditions of the multiple cache partitions; the expansion-needed partition is a cache partition with resource requirements, and the target partition is a cache partition that supports resource migration; A pre-migration module, configured to determine to migrate at least a part of the resources of the target partition to the expansion-needed partition.

[0007] The present application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above cache partition resource scheduling methods when executing the computer program.

[0008] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program implements the steps of any of the above cache partition resource scheduling methods when executed by a processor.

[0009] The present application also provides a computer program product including a computer program, which implements the steps of any of the above cache partition resource scheduling methods when executed by a processor.

[0010] The present application first obtains the resource usage of multiple cache partitions, understands the actual resource usage status of each partition in real time, and provides an accurate basis for subsequent resource adjustment; according to the obtained resource usage, it determines the partition to be expanded and the target partition. The determination of the partition to be expanded can accurately meet the partitions with resource requirements, avoiding the problem of insufficient resources caused by fixed reservation in the traditional scheme; at the same time, it is clear that the target partition is the partition that supports resource migration, and it can find the partition with idle resources available for allocation, changing the situation where idle resources cannot be effectively utilized in the traditional scheme; at least a part of the resources of the target partition is migrated to the partition to be expanded, and through this dynamic resource migration method, the redistribution of cache resources is realized. Therefore, compared with the isolation of reserved resources in each partition in the related art, the present application supports the dynamic transfer of resources across partitions, uses the resource usage of each partition as the scheduling basis, makes the resource allocation match the actual load, and avoids resource limitation; it allocates resources in advance through predictive scheduling instead of waiting for resource shortage and then making passive adjustments, realizing the dynamic balance and efficient utilization of cache resources. Description of the Drawings

[0011] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic diagram of the transfer logic for storing data; Figure 2 It is a schematic flowchart of a method for scheduling cache partition resources provided by an embodiment of the present application; Figure 3 It is a schematic diagram of the resource usage of multiple cache partitions provided by an embodiment of the present application; Figure 4Schematic diagram of resource usage of the partition to be expanded provided by the embodiment of the present application; Figure 5 Flow schematic diagram of another method for scheduling cache partition resources provided by the embodiment of the present application; Figure 6 Structural schematic diagram of a device for scheduling cache partition resources provided by the embodiment of the present application; Figure 7 Structural schematic diagram of an electronic device provided by the embodiment of the present application. Detailed implementation manners

[0013] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0014] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0015] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the technical terms used in the embodiments: The cache module is a component in a computer system used to temporarily store data. By its fast access characteristics, it reduces the direct access to the slow backend storage (such as a disk), thereby improving the data reading efficiency.

[0016] The cache partition is to divide the cache module into multiple independent regions, and each partition can manage the data caching policy separately. Each partition of the cache module will be allocated a fixed amount of resources during system initialization. The resources are used to ensure that the data in the cache can be restored after power failure, and the resource application is in units of pages.

[0017] A disk array is a storage unit composed of multiple physical disks combined by hardware or software. Through technologies such as data redundancy and striping, it improves storage performance, reliability or capacity. Each cache partition corresponds to a backend disk array.

[0018] As Figure 1 shown, Figure 1It is a schematic diagram of the transfer logic for storing data. When the host initiates data reading and writing (IO), it first reaches the cache module. The cache module distributes data reading and writing to different cache partitions for processing according to certain rules, so that different IOs do not interfere with each other. After the cache partition processes the data, it is persisted to the corresponding hard disk array to ensure that the data is not lost.

[0019] A solid state drive (SSD) stores data using solid state electronic storage chips (such as flash memory chips) and has no mechanical moving parts. It features fast read and write speeds, good shock resistance, low power consumption, and no noise, which can effectively improve the data processing efficiency of the device.

[0020] A hard disk drive (HDD) reads and writes data by means of rotating magnetic disks and moving magnetic heads, and has mechanical components such as disks, magnetic heads, and motors inside. It has a large capacity and low cost, and is suitable for storing a large amount of data.

[0021] A hybrid hard drive (HHD) combines the magnetic storage of a hard disk drive and the high-speed cache of flash memory, integrating the advantages of a large capacity and low cost of a mechanical disk and the acceleration of common data reading and writing by flash memory.

[0022] To enable those skilled in the art of this technology to better understand the solution of this application, the following further elaborates on this application in conjunction with the accompanying drawings and specific implementation manners.

[0023] In combination with the specific application environment architecture or specific hardware architecture on which the execution of the scheduling method of cache partition resources depends, the specific application environment architecture or specific hardware architecture is described herein.

[0024] The core environmental characteristics of the application environment architecture include: distributed cache cluster architecture, hierarchical cache architecture, and containerization / cloud native environment adaptation. Among them, the cache system consists of a cluster of multiple physical nodes or virtual nodes, and each node bears several cache partitions. The partition mapping relationship, resource quotas, and status information are maintained through a central controller or a distributed coordination service. High-speed network connections are required between nodes to support real-time migration of resources. The scheduling method needs to sense the resource pressure of each layer of cache, and based on the analysis of data access frequency, preferentially deploy hot data partitions on the high-performance cache layer and migrate cold data partitions to the capacity-based storage layer. Based on the container orchestration platform, cache partitions can be mapped to container group resources, and resource quotas can be dynamically adjusted.

[0025] The hardware architecture includes computing and storage hardware, hardware acceleration components, and a hardware monitoring module. Among them, the computing and storage hardware includes a central processing unit, memory, storage media, and network devices. The hardware acceleration components include an accelerator and a smart network card. The accelerator is used to offload compute-intensive tasks such as cache data compression and encryption, releasing the resources of the central processing unit (CPU) for scheduling logic. The smart network card supports direct writing of network data into memory, reducing the CPU interruption overhead during resource migration. The hardware monitoring module includes a resource monitoring sensor and performance counters. Temperature, voltage, and memory usage sensors integrated on the server motherboard provide real-time feedback on the health status of the hardware. The performance counters record metrics such as cache hit rate and memory bandwidth utilization, providing data support for scheduling decisions.

[0026] Embodiments of the present application provide a method for scheduling cache partition resources. The method will be described in detail in combination with the execution process of the method for scheduling cache partition resources.

[0027] As Figure 2 shown, Figure 2 is a schematic flowchart of a method for scheduling cache partition resources provided by an embodiment of the present application. The method includes the following steps S201 to S203.

[0028] S201. Obtain the resource usage conditions of multiple cache partitions.

[0029] The multiple cache partitions are obtained by dividing the cache module according to specific rules. The resource usage conditions include at least one of the resource utilization rate, whether there are idle resources, the current maximum available resource number, and the number of read / write operations to be allocated. Among them, the read / write operations to be allocated are read / write operations waiting for resources. As Figure 3 shown, the resource usage conditions of the multiple cache partitions involve the initial available maximum resource number of the partition, the target resource number of the partition, the current maximum available resource number of the partition, the idle resource number of the partition, and the used resource number of the partition. Among them, the initial available maximum resource number of the partition represents the maximum resources initially allocated to the partition; the target resource number of the partition represents the resources that the partition is expected to finally reach; the current maximum available resource number of the partition represents the maximum resources that can actually be called by the partition; the idle resource number of the partition represents the idle resources; the used resource number of the partition represents the resources that have been occupied. In the figure, "the current available maximum resource number of the partition = the idle resource number of the partition + the used resource number of the partition" means that the used resource number of the partition can be released, making resource allocation more flexible and available.

[0030] In some embodiments, before step S201, the method further includes: obtaining the total maximum available resources of multiple cache partitions, and allocating an initial maximum available resource number for the multiple cache partitions according to the total maximum available resources and the number of the multiple cache partitions. The total maximum available resources are the total resources that the cache module can use at most during system initialization.

[0031] It can be understood that the total maximum available resources of the cache module are obtained, and then the initial value of the maximum available resource number of each cache partition is calculated according to the total maximum available resources and the number of cache partitions, so as to initialize the maximum available resource number of each cache partition. Thus, the total resources are evenly distributed to each partition to avoid unbalanced resource allocation.

[0032] Optionally, in the process of determining the initial maximum available resource number of each cache partition according to the total maximum available resources and the number of multiple cache partitions, it includes: first calculating the quotient and remainder of the total maximum available resources divided by the number of multiple cache partitions. Then, the sum of the quotient and the remainder is used as the initial maximum available resource number of the first cache partition among the multiple cache partitions, and the quotient is used as the initial maximum available resource number of the other cache partitions except the first cache partition. The quotient is the basic resource number allocated to each cache partition by default, and the remainder is the remaining resources after the basic resource allocation.

[0033] It can be understood that dividing the total maximum available number by the number of partitions gives a quotient or a quotient and a remainder. If there is a remainder, the sum of the quotient and the remainder is used as the initial maximum available resource number of the first cache partition, and the initial maximum available resource number of the other cache partitions is equal to the quotient. If there is no remainder, the initial maximum available resource number of each cache partition is the same and equal to the quotient.

[0034] As an alternative implementation provided by the embodiments of the present application, after calculating the quotient and remainder of the total maximum available resources divided by the number of multiple cache partitions, the remainder can be evenly distributed to some cache partitions. Exemplarily, assume there are N cache partitions, the quotient of the total maximum available resources divided by N is denoted as Q, and the remainder is denoted as R. Q resources are allocated to N - R cache partitions, and the remaining R resources are evenly distributed to R partitions.

[0035] The above embodiments ensure that there is no remaining in the initial allocation of total resources, make full use of all available resources, and achieve fast and balanced allocation of resources during system initialization.

[0036] In some embodiments, the resource usage of multiple cache partitions is periodically counted by a timer. The resource usage of the multiple cache partitions includes any one of the following: (1) the probability value that meets a preset condition within a historical period is greater than a preset probability value, where the preset condition includes that the resource usage rate is greater than a preset usage rate and the first quantity of read / write operations to be allocated is greater than or equal to a first threshold; (2) the current maximum available resource quantity is greater than or equal to a preset resource quantity and there are idle resources; (3) the current maximum available resource quantity is greater than or equal to a preset resource quantity, there are no idle resources, but the second quantity of read / write operations to be allocated is less than or equal to a second threshold; (4) the current maximum available resource quantity is less than a preset resource quantity.

[0037] Continuously monitoring the resource usage in the above embodiments is beneficial to real-time adjustment of resource allocation.

[0038] S202. Determine the partition to be expanded and the target partition according to the resource usage of multiple cache partitions.

[0039] Among them, the partition to be expanded is a cache partition with resource requirements and is the partition that needs to expand the resource quantity. The target partition is a cache partition that supports resource migration, indicating that the resources it owns can be lent to other cache partitions. The partition to be expanded and the target partition are two types of cache partitions among multiple cache partitions. This application also includes other types of cache partitions, such as resource private partitions. The resource private partition means that the resources of this cache partition cannot be lent to other cache partitions.

[0040] For ease of description, the process of determining the cache partition type will be introduced later using any two different cache partitions among multiple cache partitions, such as the first cache partition and the second cache partition.

[0041] In some embodiments, determining the partition to be expanded according to the resource usage of multiple cache partitions includes: calculating the probability value that the first cache partition meets the preset condition within a historical period, where the preset condition includes that the resource usage rate is greater than the preset usage rate and the first quantity of read / write operations to be allocated is greater than or equal to the first threshold; and determining the first cache partition as the partition to be expanded when the probability value is greater than the preset probability value.

[0042] When the resource utilization rate in the preset conditions is greater than the preset utilization rate, it indicates that the core resources of the first cache partition are occupied and the free resources are limited. An exemplary preset utilization rate can be 100%. The first quantity of read / write operations to be allocated is greater than or equal to the first threshold, indicating that when the resource utilization rate is saturated, new read / write operations cannot be processed immediately and will enter the waiting queue. If the number of read / write operations waiting to be processed exceeds the first threshold, it means that the resource bottleneck has caused read / write backlog and blockage. The probability value of meeting the preset conditions within the historical period is greater than the preset probability value, indicating that the above situation occurs frequently within the historical period, indicating that resource shortage has become the norm. Based on this, it can be determined that the first cache partition is a cache partition with resource requirements, that is, the partition to be expanded.

[0043] Specifically, for the first cache partition, first determine whether its resource utilization rate is greater than the preset utilization rate within the historical period, and whether the first quantity of read / write operations to be allocated is greater than or equal to the first threshold. If so, it means that the resource utilization is saturated and the read / write operations to be allocated are waiting in excess, then count whether the probability value of the above conditions occurring within the historical period is greater than the preset probability value. If the probability of the above conditions occurring within the historical period is greater than the preset probability value, excluding occasional occurrences, it is determined that this first cache partition is the partition to be expanded.

[0044] Exemplarily, such as Figure 4 The number of resources in the partition to be expanded, that is, the aforementioned partition to be expanded, involves the maximum available resources at partition initialization, the target resources of the partition, the current maximum available resources of the partition, the free resources of the partition, and the used resources of the partition. Among them, "the free resources of the partition = 0" means that there are no unrestricted resources in the partition to be expanded and all resources have been occupied; "the current maximum available resources of the partition = the used resources of the partition" means that because there are no free resources, the current available upper limit is equal to the used resource amount.

[0045] The above embodiments determine that the cache partition is the partition to be expanded by the situation of resource saturation in the cache partition and continuous impact on the service, avoiding blind expansion; judging the normalization problem through historical probability, avoiding system crashes caused by long-term resource shortages, and realizing the accurate positioning of resource bottlenecks.

[0046] In some embodiments, according to the resource usage conditions of multiple cache partitions, the target partition is determined, including: obtaining the current maximum available resources of the second cache partition, and judging whether there are free resources in the second cache partition when the current maximum available resources are greater than or equal to the preset resources; if there are free resources in the second cache partition, determine the second cache partition as the target partition. Optionally, the preset resources are calculated according to the preset coefficient and the initial maximum available resources. For example, the preset resources are 10% of the initial maximum available resources.

[0047] Specifically, it is determined whether the current maximum available resources of the second cache partition are greater than or equal to a preset resource number. If so, it can be ensured that there are still basic available resources after the resources of the second cache partition are lent out, and further determine whether there are idle resources in the second cache partition. The existence of idle resources in the second cache partition indicates that there is a resource surplus that can be lent out in the second storage partition, which is a prerequisite for the second cache partition to lend out resources. It is determined that the second cache partition supports resource migration, that is, the second cache partition is the target partition.

[0048] Through the dual verification of the preset resource number and the idle resources in the above embodiments, it is ensured that there are still enough idle resources after the resources of the second cache partition are lent out, avoiding insufficient resources of itself due to lending out resources. This judgment method locates the target partition where there are idle resources that can be migrated and lent out to other cache partitions, which is beneficial to realizing the dynamic scheduling of resources among cache partitions and improving resource utilization. Compared with the traditional technology, it meets the resource fluctuation requirements of multiple partitions through dynamic resource migration.

[0049] In some other embodiments, according to the resource usage conditions of multiple cache partitions, determining the target partition further includes: if there are no idle resources in the second cache partition, then determine whether the second quantity of the read-write operations to be allocated in the historical period of the second cache partition is less than or equal to a second threshold; in the case where the second quantity is less than or equal to the second threshold, determine the second cache partition as the target partition.

[0050] Specifically, the current maximum available resources of the second cache partition are greater than or equal to the preset resource number, but there are no idle resources, indicating that the resources of the second cache partition have been fully occupied. However, the second quantity of the read-write operations to be allocated in the historical period is less than or equal to the second threshold, indicating that the resource occupation does not cause read-write blockage. In this case of resource usage, although the second cache partition has no idle resources, it will not cause read-write blockage due to insufficient resources, indicating that the actual processing efficiency of the second cache partition is high, and lending out some resources is safe and will not affect its performance, and it can be used as the target partition.

[0051] The above embodiments are applicable to business scenarios with intensive resources but low waiting. By verifying historical data, it can be determined to use the cache partition with saturated but efficient resources as the target partition, which is beneficial to avoiding resource waste and ensuring the security of resource migration.

[0052] In some embodiments, according to the resource usage conditions of multiple cache partitions, the resource private partition can also be determined, including: obtaining the current maximum available resource number of the second cache partition, and in the case where the current maximum available resource number of the second cache partition is less than or equal to the preset resource number, determining the second cache partition as the resource private partition, indicating that the resources of the second cache partition cannot be migrated to any other cache partition.

[0053] In the above embodiments, when the current maximum available resources in the cache partition are less than or equal to the preset resources, it indicates that the cache partition is in a resource - tight state. Borrowing resources will lead to resource exhaustion and will not be able to cope with possible sudden requests and business fluctuations. Therefore, it is marked as a resource - private partition, which is to ensure the basic availability of the cache partition and is conducive to balancing the global and local resource allocation to a certain extent.

[0054] S203. Determine to migrate at least a part of the resources of the target partition to the partition to be expanded.

[0055] If the target partition is different, at least a part of the resources migrated to the partition to be expanded is different.

[0056] In some embodiments, if there are idle resources in the target partition, determine to migrate the idle resources to the partition to be expanded. If there are idle resources in the target partition, the idle resources can be migrated to the partition to be expanded with resource requirements, avoiding waste of resource fragmentation, realizing dynamic resource scheduling, and improving the overall utilization rate.

[0057] Based on the above embodiments, if there are idle resources in the target partition, determine the number of idle resources according to this part of idle resources, and then update the current maximum available resources according to the current maximum available resources and the number of idle resources in the target partition.

[0058] Specifically, the new current maximum available resources = current maximum available resources - number of idle resources. By modifying the upper limit of available resources in the target cache partition, it prompts the system to schedule idle resources to the partition to be expanded, and also restricts subsequent read - write operation requests for resources, avoiding excessive resource waste or restriction, and improving the global resource utilization rate.

[0059] In some embodiments, if there are no idle resources in the target partition, determine the target available resources according to the preset coefficient and the current maximum available resources. Then, determine the number of resources to be released according to the current maximum available resources and the target available resources. Furthermore, determine to migrate the resources to be released corresponding to the number of resources to be released to the partition to be expanded. The preset coefficient can be 90%.

[0060] Specifically, if there are no idle resources in the target partition, it indicates that the resources in the target partition are being occupied by read - write operations. Calculate the product of the preset coefficient and the current maximum available resources to obtain the target available resources. Calculate the difference between the current maximum available resources and the target available resources to obtain the number of resources to be released. After the read - write operation is completed, the resources will be released. In this application, the resources will not be directly returned to the target partition, but determine to migrate this part of the resources to be released to the partition to be expanded, avoiding the situation where resources are physically occupied but logically idle. Wait for the resources to be released and then migrate them to the partition to be expanded to avoid interrupting the ongoing read - write operations. In the above embodiments, resources are migrated from the target partition to the partition to be expanded. Without pre - allocating fixed reserved space, through the dynamic migration mechanism, the system can flexibly adjust resources according to actual needs without manual intervention, thus achieving the dynamic balance of cache resources, improving the overall resource utilization rate of the system, and better meeting the requirements of the storage system for optimized resource management.

[0061] Based on the above embodiments, update the current maximum available resources of the target partition according to the target available resources. Optionally, during the process of updating the current maximum available resources of the target partition, as the resources released by the pending read - write operations are counted, gradually deduct the current maximum available resources until they are equal to the target available resources. Set the current maximum available resources as the target resources. After migrating the resources to be released, the target partition still has enough resources to handle read - write operations, which not only meets the needs of other partitions but also ensures the stability of the target partition itself.

[0062] The above embodiments dynamically adjust the available upper limit of partition resources. On the premise of ensuring the IO processing capacity of a single cache partition, they achieve the efficient scheduling and balance of global resources, avoid resource idleness or unbalanced allocation, and are conducive to improving the overall performance and stability of the storage system.

[0063] In summary, the present application first obtains the resource usage of multiple cache partitions. Different from the fixed reserved space in the traditional scheme, it can understand the actual resource usage status of each partition in real - time, providing an accurate basis for subsequent resource adjustment, which helps to discover the idle resources in low - demand partitions and the resource shortage in high - demand partitions; according to the obtained resource usage, determine the partition to be expanded and the target partition. The determination of the partition to be expanded can accurately meet the partitions with resource requirements, avoiding the problem of resource shortage caused by fixed reservation in the traditional scheme. At the same time, by clarifying the target partition as the partition that supports resource migration, it can find the partition with idle resources available for allocation, changing the situation where idle resources cannot be effectively utilized in the traditional scheme; migrate at least a part of the resources of the target partition to the partition to be expanded. Through this dynamic resource migration method, the re - allocation of cache resources is realized. Compared with the fixed resource reservation mode in the traditional scheme, this method can flexibly adjust resources according to actual needs, making resources flow from partitions with low utilization rate to partitions with high utilization rate, thus achieving the dynamic balance of cache resources, improving the overall resource utilization rate of the system, and better meeting the requirements of the storage system for optimized resource management. The scheduling method of cache partition resources realizes the balanced allocation of resources through dynamic resource migration, effectively solving the problems of the traditional reserved space scheme.

[0064] Another embodiment of the present application provides a scheduling method for cache partition resources, as Figure 5 shown. This method includes the following steps S501 - S505: S501. Obtain the resource usage of multiple cache partitions.

[0065] S502. Determine the partition to be expanded and the target partition according to the resource usage of multiple cache partitions.

[0066] For the specific implementation of the above steps, reference can be made to the foregoing steps S201~S202, and details are not described herein in this application.

[0067] S503. In the case where it is determined according to the resource usage of multiple cache partitions that there is a partition to be expanded but no target partition, load the data required for the read / write operations to be allocated in the partition to be expanded into the partition to be expanded.

[0068] Among them, the read / write operations to be allocated in the partition to be expanded are triggered in response to the read / write requests submitted by the host.

[0069] S504. Synchronize the data required for the read / write operations to be allocated stored in the partition to be expanded to the disk array.

[0070] S505. After receiving the response returned by the disk array, return a response indicating the completion of the read / write operations to be allocated to the host.

[0071] In the case where it is determined according to the resource usage of multiple cache partitions that there is a partition to be expanded but no target partition, it means that the resource utilization rate of all cache partitions is almost saturated, and the global resource tension problem cannot be solved through internal resource scheduling. For such a problem, in this application, the data required for the read / write operations to be allocated in the partition to be expanded is first loaded into the partition to be expanded, and then these data are flushed down to the disk array and wait for the response of the disk array. After the disk array responds, a response indicating the completion of the read / write operations to be allocated is returned to the host. Compared with the traditional read / write operation processing flow where "host response time = partition processing time", in this application, after waiting for the disk array to respond and then replying to the host, "host response time = partition processing time + disk response time", which lengthens the time of the IO processing link, making the host perceive that the IO processing becomes slower, thereby reducing the number of IOs submitted per unit time.

[0072] The above embodiments can reduce the rate of IOs issued by the host by extending the read / write operation processing time, realizing the interaction between the storage and the host, alleviating the storage resource pressure, and avoiding cache overload.

[0073] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0074] Such as Figure 6As shown in the figure, an embodiment of the present application further provides a scheduling device for cache partition resources, and the device includes: An acquisition module 601, configured to acquire the resource usage of multiple cache partitions; A type definition module 602, configured to determine an expansion-required partition and a target partition according to the resource usage of multiple cache partitions; the expansion-required partition is a cache partition with resource requirements, and the target partition is a cache partition that supports resource migration; A pre-migration module 603, configured to determine to migrate at least a part of the resources of the target partition to the expansion-required partition.

[0075] As an optional implementation manner of the embodiment of the present application, the type definition module 602 is specifically configured to: calculate a probability value of a first cache partition meeting a preset condition within a historical period; wherein, the first cache partition is any one of the multiple cache partitions, and the preset condition includes that the resource usage rate is greater than a preset usage rate and the first quantity of read / write operations to be allocated is greater than or equal to a first threshold; when the probability value is greater than a preset probability value, determine that the first cache partition is the expansion-required partition.

[0076] As an optional implementation manner of the embodiment of the present application, the type definition module 602 is specifically configured to: acquire the current maximum available resources of a second cache partition; the second cache partition is any one of the multiple cache partitions; when the current maximum available resources are greater than or equal to the preset resources, determine whether the second cache partition has idle resources; if the second cache partition has idle resources, determine that the second cache partition is the target partition; if the second cache partition does not have idle resources, determine whether the second quantity of read / write operations to be allocated by the second cache partition within a historical period is less than or equal to a second threshold; when the second quantity is less than or equal to the second threshold, determine that the second cache partition is the target partition.

[0077] As an optional implementation manner of the embodiment of the present application, the pre-migration module 603 is specifically configured to: if the target partition has idle resources, determine to migrate the idle resources to the expansion-required partition.

[0078] As an optional implementation manner of the embodiment of the present application, the device further includes an update module, configured to: if the target partition has idle resources, determine the number of idle resources according to the idle resources existing in the target partition; update the current maximum available resources according to the current maximum available resources of the target partition and the number of idle resources.

[0079] As an alternative implementation manner of the embodiment of the present application, the pre-migration module 603 is specifically configured to: if there is no idle resource in the target partition, determine the target available resource number according to a preset coefficient and the current maximum available resource number; determine the resource number to be released according to the current maximum available resource number and the target available resource number; determine to migrate the resource to be released corresponding to the resource number to be released to the partition to be expanded.

[0080] As an alternative implementation manner of the embodiment of the present application, the device further includes an update module, configured to: update the current maximum available resource number according to the target available resource number.

[0081] As an alternative implementation manner of the embodiment of the present application, the device further includes a processing module, configured to: in the case that there is a partition to be expanded determined according to the resource usage of multiple cache partitions, but there is no target partition, load the data required for the read / write operation to be allocated in the partition to be expanded into the partition to be expanded; synchronize the data required for the read / write operation to be allocated stored in the partition to be expanded to the disk array; after receiving the response returned by the disk array, return a response indicating the completion of the read / write operation to be allocated to the host.

[0082] As an alternative implementation manner of the embodiment of the present application, the type definition module 602 is further configured to: in the case that the current maximum available resource number is less than the preset resource number, determine that the second cache partition is a resource private partition, indicating that the resources of the second cache partition cannot be migrated to any other cache partition.

[0083] As an alternative implementation manner of the embodiment of the present application, the device further includes an initialization module, configured to: obtain the total maximum available resources of multiple cache partitions; allocate an initial maximum available resource number for multiple cache partitions according to the total maximum available resources and the number of multiple cache partitions.

[0084] As an alternative implementation manner of the embodiment of the present application, the initialization module is specifically configured to: calculate the quotient and remainder of the total maximum available resources and the number of multiple cache partitions; use the sum of the quotient and the remainder as the initial maximum available resource number of the first cache partition among multiple cache partitions, and use the quotient as the initial maximum available resource number of other cache partitions except the first cache partition.

[0085] For the description of the features in the corresponding embodiment of the cache partition resource scheduling device, reference can be made to the relevant description in the corresponding embodiment of the cache partition resource scheduling method, which will not be elaborated here one by one.

[0086] An embodiment of the present application further provides an electronic device, as Figure 7 shown, including a memory 701 and a processor 702. A computer program is stored in the memory 701, and the processor 702 is configured to run the computer program to execute the steps in any of the above embodiments of the cache partition resource scheduling method.

[0087] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above-described embodiments of the scheduling method for cache partition resources when running.

[0088] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memories (ROM), random access memories (RAM), mobile hard disks, magnetic disks, or optical discs that can store computer programs.

[0089] An embodiment of the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the scheduling method for cache partition resources.

[0090] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above-described embodiments of the scheduling method for cache partition resources.

[0091] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0092] The above has introduced in detail a scheduling method, device, electronic device, medium, and product for cache partition resources provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A scheduling method for cache partition resources, characterized in that Including: Obtain the resource usage of multiple cache partitions; Determine the partition to be expanded and the target partition according to the resource usage of the multiple cache partitions; The partition to be expanded is a cache partition with resource requirements, and the target partition is a cache partition that supports resource migration; Determine to migrate at least a part of the resources of the target partition to the partition to be expanded.

2. The method according to claim 1, wherein Determine the partition to be expanded according to the resource usage of multiple cache partitions, including: Calculate the probability value of the first cache partition meeting the preset conditions within the historical period; wherein, the first cache partition is any cache partition among the multiple cache partitions, and the preset conditions include that the resource utilization rate is greater than the preset utilization rate and the first quantity of read / write operations to be allocated is greater than or equal to the first threshold; When the probability value is greater than the preset probability value, determine the first cache partition as the partition to be expanded.

3. The method according to claim 1, characterized in that Determine the target partition according to the resource usage of multiple cache partitions, including: Obtain the current maximum available resources of the second cache partition; the second cache partition is any cache partition among the multiple cache partitions; When the current maximum available resources are greater than or equal to the preset resources, determine whether there are idle resources in the second cache partition; If there are idle resources in the second cache partition, determine the second cache partition as the target partition; If there are no idle resources in the second cache partition, determine whether the second quantity of read / write operations to be allocated in the second cache partition within the historical period is less than or equal to the second threshold; When the second quantity is less than or equal to the second threshold, determine the second cache partition as the target partition.

4. The method according to claim 3, characterized in that, The determination of migrating at least a part of the resources of the target partition to the partition to be expanded includes: If there are idle resources in the target partition, determine to migrate the idle resources to the partition to be expanded.

5. The method according to claim 3, characterized in that, The method further includes: If there are idle resources in the target partition, determine the number of idle resources according to the idle resources existing in the target partition; Update the current maximum available resources according to the current maximum available resources of the target partition and the number of idle resources.

6. The method according to claim 3, characterized in that, The determination of migrating at least a part of the resources of the target partition to the partition to be expanded includes: If there are no idle resources in the target partition, determine the target available resources according to the preset coefficient and the current maximum available resources; Determine the resources to be released according to the current maximum available resources and the target available resources; Determine to migrate the resources to be released corresponding to the resources to be released to the partition to be expanded.

7. The method according to claim 6, characterized in that, The method further includes: Update the current maximum available resources according to the target available resources.

8. The method according to claim 1, characterized in that The method further includes: In the case that there is a partition to be expanded determined according to the resource usage of the multiple cache partitions, but there is no target partition, load the data required for the read / write operations to be allocated in the partition to be expanded into the partition to be expanded; Synchronize the data required for the read / write operations to be allocated stored in the partition to be expanded to the disk array; After receiving the response returned by the disk array, return a response indicating the completion of the read / write operations to be allocated to the host.

9. The method according to claim 3, wherein The method further includes: In the case that the current maximum available resource number is less than the preset resource number, determine that the second cache partition is a resource private partition, indicating that the resources of the second cache partition cannot be migrated to any other cache partition.

10. The method according to claim 1, wherein Before obtaining the resource usage of multiple cache partitions, the method further includes: Obtain the total maximum available resources of the multiple cache partitions; Allocate an initial maximum available resource number for the multiple cache partitions according to the total maximum available resources and the number of the multiple cache partitions.

11. The method according to claim 10, characterized in that The allocating an initial maximum available resource number for the multiple cache partitions according to the total maximum available resources and the number of the multiple cache partitions includes: Calculate the quotient and remainder of the total maximum available resources and the number of the multiple cache partitions; Use the sum of the quotient and the remainder as the initial maximum available resource number of the first cache partition among the multiple cache partitions, and use the quotient as the initial maximum available resource number of the other cache partitions except the first cache partition.

12. A scheduling device for cache partition resources, characterized in that, Includes: An obtaining module, configured to obtain the resource usage of multiple cache partitions; A type defining module, configured to determine a partition to be expanded and a target partition according to the resource usage of the multiple cache partitions; The partition to be expanded is a cache partition with resource requirements, and the target partition is a cache partition that supports resource migration; A pre-migration module, configured to determine to migrate at least a part of the resources of the target partition to the partition to be expanded.

13. An electronic device, characterized in that, Includes: A memory, configured to store a computer program; A processor, configured to implement the steps of the scheduling method for cache partition resources as described in any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the scheduling method for cache partition resources as described in any one of claims 1 to 11 when executed by a processor.

15. A computer program product comprising a computer program, characterized in that, The computer program implements the steps of the scheduling method for cache partition resources as described in any one of claims 1 to 11 when executed by a processor.

Images