Cache partition dynamic allocation method and device, equipment and storage medium

Abstract

The invention discloses a cache partition dynamic allocation method and device, equipment and a storage medium, and relates to the technical field of storage, and the method comprises the following steps: obtaining an initial cache hit rate and an initial data transmission bandwidth of each I / O data stream; adjusting cache resources corresponding to the I / O data stream for multiple times according to a preset cache step length adjustment rule, and obtaining an adjusted cache hit rate and a data transmission bandwidth; according to the initial cache hit rate, the initial data transmission bandwidth, the adjusted cache hit rate, the adjusted data transmission bandwidth and the cache resource adjustment vector, generating cache sensitivity; after all the cache sensitivities are obtained, sorting all the cache sensitivities; and according to the sorting result and a pre-configured cache resource adjustment rule, determining to dynamically adjust the cache resources of part or all of the I / O data streams. According to the method, performance reduction caused by cache pollution and resource competition is effectively reduced, and cache resources are utilized more efficiently.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for dynamic allocation of cache partitions. Background Technology

[0002] In multi-core solid-state drives (SSDs), the onboard DRAM cache is typically used as a shared cache, which can be used by any I / O data stream. However, this can lead to cache pollution. Traditional cache partitioning methods include shared mode and even-sharing mode. When the random workload of I / O data streams is heavy, shared cache may experience contention among multiple different data streams, resulting in high-utilization cache data not being able to obtain resources and causing performance degradation. In even-sharing mode, if an I / O data stream does not fully utilize its cache partition, insufficient cache utilization will occur, while other I / O data streams may experience cache resource shortages. Summary of the Invention

[0003] This application provides a method, apparatus, device, and storage medium for dynamic allocation of cache partitions, in order to at least solve the problem of resource contention caused by cache pollution due to shared cache in multi-core SSDs, which leads to performance degradation.

[0004] This application provides a method for dynamically allocating cache partitions, including:

[0005] After the cache resources corresponding to each I / O data stream of the multi-core dynamic hard disk have been initialized and configured, the initial cache hit rate and initial data transfer bandwidth of each I / O data stream during the operation are obtained.

[0006] The cache resources corresponding to each I / O data stream are adjusted multiple times according to the preset cache step size adjustment rules, and the cache hit rate and data transmission bandwidth after each adjustment are obtained.

[0007] Based on the initial cache hit rate and initial data transfer bandwidth corresponding to the first I / O data stream, the cache hit rate and data transfer bandwidth after each adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment, a cache sensitivity corresponding to the first I / O data stream is generated. The first I / O data stream is any I / O data stream.

[0008] After obtaining the cache sensitivity of each I / O data stream, sort the cache sensitivity of each I / O data stream.

[0009] Based on the sorting results and the pre-configured cache resource adjustment rules, determine whether to dynamically adjust the cache resources corresponding to some or all I / O data streams.

[0010] This application also provides a dynamic cache partition allocation device, including:

[0011] The acquisition module is used to acquire the initial cache hit rate and initial data transfer bandwidth of each I / O data stream during operation after the cache resources corresponding to each I / O data stream of the multi-core dynamic hard disk have been initialized and configured.

[0012] The first adjustment module is used to adjust the cache resources corresponding to each I / O data stream multiple times according to the preset cache step size adjustment rules;

[0013] The acquisition module is also used to acquire the adjusted cache hit rate and adjusted data transmission bandwidth for each adjustment.

[0014] The processing module is used to generate a cache sensitivity corresponding to the first I / O data stream based on the initial cache hit rate and initial data transmission bandwidth corresponding to the first I / O data stream, the cache hit rate and data transmission bandwidth after each adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment. The first I / O data stream can be any I / O data stream.

[0015] The sorting module is used to sort the cache sensitivities of all I / O data streams after obtaining them.

[0016] The second adjustment module is used to determine the dynamic adjustment of cache resources corresponding to some or all I / O data streams based on the sorting results and pre-configured cache resource adjustment rules.

[0017] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described methods for dynamic allocation of cache partitions.

[0018] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described methods for dynamic allocation of cache partitions.

[0019] This application first obtains the initial cache hit rate and initial data transfer bandwidth of the I / O data stream after initialization. Then, according to a preset cache step size adjustment rule, the cache resources corresponding to each I / O data stream are adjusted multiple times. The cache hit rate and adjusted data transfer bandwidth are then obtained after each adjustment. Based on these parameters, the cache sensitivity of each I / O data stream is calculated. The cache partitions of each data stream are then dynamically adjusted according to its cache sensitivity, and higher priority is given to data streams that benefit most from the increased cache allocation. By dynamically adjusting partitions based on cache sensitivity, performance degradation caused by cache pollution and resource contention is effectively reduced. Furthermore, this application's embodiments can accurately allocate cache resources based on the cache sensitivity of different data streams. Partitions are increased for cache-sensitive data streams, while partitions are reduced for insensitive data streams with sufficient capacity. This avoids the contention problems of traditional shared caches and the unreasonable utilization of equally partitioned caches, resulting in more efficient use of cache resources. Dynamically adjusting cache partitions allows each data stream to more stably obtain appropriate cache resources, reducing performance fluctuations caused by insufficient or excessive cache usage, and improving system stability and reliability. Attached Figure Description

[0020] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This application provides a schematic flowchart of a method for dynamically allocating cache partitions according to an embodiment of the present application.

[0022] Figure 2 A schematic diagram of another method for dynamically allocating cache partitions provided in this application embodiment;

[0023] Figure 3 A simplified diagram illustrating the cache resource management process when multiple rounds of cyclical adjustments are present, as provided in this application.

[0024] Figure 4 A simplified structural diagram illustrating the overall flow of the cache partitioning implementation method provided in this application;

[0025] Figure 5 A schematic diagram of a cache partition dynamic allocation device provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0028] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0029] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] SSDs, due to their high reliability, low power consumption, ever-increasing capacity, and decreasing cost, hold a crucial position in both consumer electronics and enterprise markets, serving as the primary storage medium in modern data centers. To further enhance SSD performance and lifespan, a smaller but faster Dynamic Random Access Memory (DRAM) module is typically incorporated into the SSD as a data cache. The cache handles data reading and writing, and also functions as a cache mapping table. When a user's I / O request hits the cache, the data access can be absorbed by the cache, reducing operations on flash memory. When the cache is not full, its use significantly improves data write performance. However, under heavy workloads, the cache becomes exhausted, and throughput drops sharply due to the performance difference between DRAM and flash memory. To further improve cache hit rates, typically only a small portion of frequently accessed data is placed in the cache.

[0031] Based on this characteristic, different cache management methods exist. These different cache management methods determine the efficiency of DRAM data caching and the read / write performance of SSDs.

[0032] To improve SSD read / write performance, multi-core processing frameworks can be used in related technologies. Multiple cores in an SSD can handle data read / write tasks simultaneously, significantly improving random read / write performance and enhancing error detection and correction capabilities. Furthermore, when one core is overloaded, other cores can take over some tasks, achieving better load balancing. Compared to traditional single-core processing where all read / write tasks are handled by a single core, which can lead to data processing delays and slower system response when handling large amounts of data, multi-core implementations offer higher read / write speeds, better random read / write performance, and better fault tolerance in applications requiring high performance and reliability.

[0033] However, in multi-core SSDs, the onboard DRAM cache is typically used as a shared cache, which can be occupied by any I / O data stream, potentially leading to cache pollution. Traditional cache partitioning methods include shared mode and even-sharing mode. When the random workload of I / O data streams is heavy, shared cache may experience contention among multiple different data streams, resulting in high-utilization cache data not being able to compete for resources, leading to performance degradation. In even-sharing mode, if an I / O data stream does not fully utilize its cache partition, insufficient cache utilization will occur, while other I / O data streams may experience cache resource shortages.

[0034] To address the aforementioned issues, embodiments of this application provide a method for dynamically allocating cache partitions, as detailed below. Figure 1 As shown, the method includes the following steps:

[0035] Step S101: After the cache resources corresponding to each I / O data stream of the multi-core dynamic hard disk have been initialized and configured, the initial cache hit rate and initial data transfer bandwidth of each I / O data stream during operation are obtained.

[0036] Specifically, initialization configuration operations are performed on the cache resources corresponding to the I / O data streams of the multi-core dynamic hard disk SSD, including:

[0037] The cache resources are allocated evenly according to the total number of all I / O data streams. In other words, the initial cache resources for each I / O data stream are the same, ensuring that each I / O data stream starts running under the same cache conditions.

[0038] After running for a period of time and determining that the working state of the I / O data stream is stable, such as that the cache hit rate and data transfer bandwidth are no longer changing, or the changes are within a certain controllable range, the initial cache hit rate and initial data transfer bandwidth of each I / O data stream during the operation are obtained.

[0039] Step S102: Adjust the cache resources corresponding to each I / O data stream multiple times according to the preset cache step size adjustment rules, and obtain the cache hit rate and data transmission bandwidth after each adjustment.

[0040] Specific preset cache step size adjustment rules include, for example, multiple cache resource adjustment ranges and corresponding adjustment directions for each range. For each I / O data stream, adjustments are made sequentially according to each cache resource adjustment range and its corresponding direction. After each adjustment, the cache hit rate and data transfer bandwidth are obtained. Specific adjustment directions include increasing or decreasing the cache resource adjustment range. The cache resource adjustment range can be a multiple of the preset cache step size. In other words, the preset cache step size adjustment rules include increasing or decreasing the cache resources corresponding to the I / O data stream by a multiple of the preset cache step size during each adjustment.

[0041] Step S103: Based on the initial cache hit rate and initial data transmission bandwidth corresponding to the first I / O data stream, the cache hit rate and data transmission bandwidth after each adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment, generate the cache sensitivity corresponding to the first I / O data stream.

[0042] Specifically, data transmission bandwidth reflects the data transmission rate of the data stream, while cache hit rate reflects the proportion of data that is hit in the cache. By comparing the initial data transmission bandwidth before adjustment with the data transmission bandwidth after each adjustment, if the adjusted data transmission bandwidth increases, it indicates that the current cache resource adjustment is beneficial. Similarly, if the cache hit rate increases, it also indicates that the current cache resource adjustment is beneficial. Conversely, if the adjusted data transmission bandwidth decreases, or the cache hit rate decreases, it may indicate that the current cache resource adjustment is detrimental to the current I / O data stream. To ensure that the cache sensitivity of the I / O data stream has a certain degree of sensitivity, the cache sensitivity can be determined by using the cache hit rate, data transmission bandwidth, initial cache hit rate, initial data transmission bandwidth, and cache resource adjustment vector obtained after multiple cache resource adjustments. The first I / O data stream is considered as one I / O data stream.

[0043] Step S104: After obtaining the cache sensitivity corresponding to each of the I / O data streams, sort the cache sensitivity corresponding to each of the I / O data streams.

[0044] Specifically, the dynamic adjustment of cache resources includes two stages. Steps 101 to 103 can be understood as the first stage of dynamic adjustment of cache resources, namely the evaluation stage.

[0045] This step marks the beginning of the second stage, namely the modification stage.

[0046] During the modification phase, after obtaining the cache sensitivity corresponding to each I / O data stream, the cache sensitivity of each I / O data stream can be sorted. The sorting result indicates the degree of impact of each I / O data stream on the adjustment of cache resources. The higher the sensitivity, the greater the impact, which can be understood as a stronger demand for cache resource adjustments. Here, the demand for cache resource adjustments typically refers to the need to increase cache resources.

[0047] Step S105: Based on the sorting results and the pre-configured cache resource adjustment rules, determine to dynamically adjust the cache resources corresponding to some or all I / O data streams respectively.

[0048] Specifically, the pre-configured cache resource adjustment rules include, for example, configuring the number of I / O data streams to be adjusted, specifically identifying which I / O data streams' cache resources need adjustment, the specific adjustment direction, and the adjustment magnitude of the cache resources. The number of I / O data streams to be adjusted mentioned in this embodiment can be partially or entirely dependent on the cache resource adjustment rules. The specific cache resource adjustment rules can be set according to actual conditions, and no restrictions are imposed here.

[0049] In a specific example, one could increase the cache resources for the first n most sensitive I / O data streams and correspondingly decrease the cache resources for the first n least sensitive I / O data streams.

[0050] This application provides a method for dynamically allocating cache partitions. First, it obtains the initial cache hit rate and initial data transfer bandwidth of the I / O data stream after initialization. Then, it adjusts the cache resources corresponding to each I / O data stream multiple times according to a preset cache step size adjustment rule. After each adjustment, it obtains the cache hit rate and adjusted data transfer bandwidth, and then calculates the cache sensitivity of each I / O data stream based on these parameters. Next, it dynamically adjusts the cache partitions of each data stream based on its cache sensitivity, assigning higher priority to data streams that benefit most from the increased cache allocation. By dynamically adjusting partitions based on cache sensitivity, it effectively reduces performance degradation caused by cache pollution and resource contention. Furthermore, this application can accurately allocate cache resources based on the cache sensitivity of different data streams. It increases partitions for cache-sensitive data streams and reduces partitions for insensitive data streams with sufficient capacity, avoiding contention problems in traditional shared caches and unreasonable utilization of equally partitioned caches, thus enabling more efficient use of cache resources. Dynamically adjusting cache partitions allows each data stream to more stably obtain appropriate cache resources, reducing performance fluctuations caused by insufficient or excessive cache usage, and improving system stability and reliability.

[0051] In an optional example, based on the foregoing embodiments, a cache sensitivity corresponding to the first I / O data stream is generated according to the initial cache hit rate and initial data transfer bandwidth corresponding to the first I / O data stream, the cache hit rate and data transfer bandwidth after each cache resource adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment. This specifically includes the following method steps, see details below. Figure 2 As shown:

[0052] Step S201: Determine the cache hit rate sensitivity for the nth time based on the initial cache hit rate, the cache hit rate after the nth adjustment, and the cache adjustment vector for the nth time.

[0053] Specifically, the cache adjustment vector for the nth adjustment includes the amount and direction of cache resource adjustment during the nth adjustment process. When the amount of cache resource adjustment is increased, the adjustment direction is positive, or when the amount of cache resource adjustment is decreased, the adjustment direction is negative, where n is a positive integer.

[0054] The cache resource adjustment amount mentioned here is the same as the cache resource adjustment range introduced earlier. The cache hit rate sensitivity for the nth adjustment, based on the initial cache hit rate, the cache path after the nth adjustment, and the cache adjustment vector, can also be expressed by the following expression:

[0055]

[0056] Among them, HR new HR is the cache hit rate after the nth adjustment. inital HS represents the initial cache hit rate, Δs represents the cache resource adjustment vector at the nth time, and HS represents the cache resource adjustment vector at the nth time. n Let n be the cache hit sensitivity.

[0057] Step S202: Determine the bandwidth sensitivity for the nth time based on the initial data transmission bandwidth, the data transmission bandwidth after the nth adjustment, and the buffer adjustment vector for the nth time.

[0058] Specifically, the cache hit sensitivity for the nth time can be represented by the following expression:

[0059]

[0060] Among them, BW new BW represents the data transmission bandwidth after the nth adjustment. inital Let BS be the initial data transmission bandwidth, Δs be the buffer resource adjustment vector for the nth iteration, and BS be the buffer bandwidth. n Let n be the bandwidth sensitivity for the nth iteration.

[0061] Step S203: Generate a cache sensitivity corresponding to the first I / O data stream based on the cache hit rate sensitivity after each cache resource adjustment and the bandwidth sensitivity after each cache resource adjustment.

[0062] Specifically, since cache command rate sensitivity and bandwidth sensitivity have different units, these two parameters can be normalized separately when calculating cache sensitivity. Then, the normalized data can be averaged to obtain the cache sensitivity.

[0063] Of course, to further reduce the amount of data computation, this method may also include the following steps:

[0064] Step a1: Determine the average cache hit rate sensitivity based on the cache hit rate sensitivity after each cache resource adjustment.

[0065] Step a2: Determine the average bandwidth sensitivity based on the bandwidth sensitivity after each cache resource adjustment.

[0066] Step a3: After normalizing the average cache hit rate sensitivity and the average bandwidth sensitivity respectively, determine the mean between the normalized average cache hit rate sensitivity and the normalized average bandwidth sensitivity, and use it as the cache sensitivity corresponding to the first I / O data stream.

[0067] Specifically, this involves calculating the average of cache hit rate sensitivity and bandwidth sensitivity after multiple cache resource adjustments. Then, only these two averages are normalized to further reduce the data volume. Finally, after normalization, the average is calculated again to obtain the final cache sensitivity.

[0068] Further, optionally, based on any of the foregoing embodiments, according to the sorting results and pre-configured cache resource adjustment rules, it is determined to dynamically adjust the cache resources corresponding to some or all I / O data streams, specifically including the following method steps:

[0069] Step b1: Increase the cache resources for the preset number of I / O data streams with the highest cache sensitivity in the sorting according to the preset adjustment step size.

[0070] Step b2: Reduce cache resources by adjusting the preset number of I / O data streams with the lowest cache sensitivity according to the preset adjustment step size.

[0071] Specifically, for example, if there are 50 I / O data streams, then the cache resources for the top 10 most sensitive I / O data streams are increased according to a preset adjustment step size. Conversely, to balance cache resources, the cache resources for the top 10 least sensitive I / O data streams need to be reduced according to a preset adjustment step size.

[0072] In one optional embodiment, the dynamic adjustment of cache resources can be periodic, thereby continuously optimizing the occupancy of cache space for different data streams in each core, making full use of cache resources, and improving performance; or it can be based on certain triggering conditions to generate a triggering mechanism. For example, in actual application, when the cache space corresponding to certain data streams is severely depleted, it indicates that a dynamic adjustment operation needs to be performed. In this way, the application of computing resources can be reduced, the impact on multiple services can be minimized, and system performance can be improved.

[0073] Therefore, the dynamic adjustment of specific cached resources can actually be determined based on the actual situation.

[0074] See details Figure 3 As shown, Figure 3 This diagram illustrates a simplified representation of the cache resource management process when multiple rounds of cyclical adjustments are involved, including several evaluation and modification phases.

[0075] Figure 4 The diagram illustrates a simplified structure of the overall process of implementing cache partitioning, including i data streams. Each stream adjusts its corresponding cache resource multiple times according to a preset cache step size adjustment rule. For example, the corresponding cache resource adjustment vector Δs for each adjustment is sequentially Δs1, Δs2, ..., Δs... nFor each I / O data stream, His (average cache hit sensitivity of the i-th I / O data stream) and Bsi (average bandwidth sensitivity of the i-th I / O data stream) are generated respectively. After calculation, the cache sensitivity of the i-th I / O data stream is finally obtained.

[0076] After sorting the cache sensitivity of all I / O data streams, it is assumed that the cache resources of the i-th and 2nd I / O data streams with the highest cache sensitivity are increased according to a preset adjustment step size; the cache resources of the 1st and (i-1th)th I / O data streams with the lowest cache sensitivity are decreased according to a preset adjustment step size. See details... Figure 4 As shown, Figure 4 This includes the entire process of adjusting cached resources.

[0077] The specific implementation process has been described in detail above, so it will not be repeated here.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0079] Embodiments of this application also provide a dynamic cache partition allocation device, see details below. Figure 5 As shown, the device includes: an acquisition module 501, a first adjustment module 502, a processing module 503, a sorting module 504, and a second adjustment module 505.

[0080] The acquisition module 501 is used to acquire the initial cache hit rate and initial data transfer bandwidth of each I / O data stream during operation after the cache resources corresponding to each I / O data stream of the multi-core dynamic hard disk have been initialized and configured.

[0081] The first adjustment module 502 is used to adjust the cache resources corresponding to each I / O data stream multiple times according to the preset cache step size adjustment rules;

[0082] The acquisition module 501 is also used to acquire the adjusted cache hit rate and adjusted data transmission bandwidth for each adjustment.

[0083] The processing module 503 is used to generate a cache sensitivity corresponding to the first I / O data stream based on the initial cache hit rate and initial data transmission bandwidth corresponding to the first I / O data stream, the cache hit rate and data transmission bandwidth after each adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment. The first I / O data stream can be any I / O data stream.

[0084] The sorting module 504 is used to sort the cache sensitivities of all I / O data streams after obtaining the cache sensitivities of each I / O data stream.

[0085] The second adjustment module 505 is used to determine, based on the sorting results and pre-configured cache resource adjustment rules, to dynamically adjust the cache resources corresponding to some or all I / O data streams.

[0086] In an optional example, the preset cache step size adjustment rule includes increasing or decreasing the cache resources corresponding to the I / O data stream by a multiple of the preset cache step size each time an adjustment is made.

[0087] In an optional example, processing module 503 is specifically used for:

[0088] The cache hit rate sensitivity for the nth adjustment is determined based on the initial cache hit rate, the cache hit rate after the nth adjustment, and the cache adjustment vector for the nth adjustment. The cache adjustment vector for the nth adjustment includes the amount and direction of cache resource adjustment during the nth adjustment process. When the amount of cache resource adjustment is increased, the adjustment direction is positive, or when the amount of cache resource adjustment is decreased, the adjustment direction is negative, where n is a positive integer.

[0089] The bandwidth sensitivity for the nth adjustment is determined based on the initial data transmission bandwidth, the data transmission bandwidth after the nth adjustment, and the buffer adjustment vector for the nth adjustment.

[0090] Based on the cache hit rate sensitivity after each cache resource adjustment and the bandwidth sensitivity after each cache resource adjustment, a cache sensitivity corresponding to the first I / O data stream is generated.

[0091] In an optional example, processing module 503 determines the cache hit rate sensitivity for the nth time based on the initial cache hit rate, the cache hit rate after the nth adjustment, and the cache adjustment vector for the nth time, expressed by the following expression:

[0092]

[0093] Among them, HR new HR is the cache hit rate after the nth adjustment. initalHS represents the initial cache hit rate, Δs represents the cache resource adjustment vector at the nth time, and HS represents the cache resource adjustment vector at the nth time. n Let n be the cache hit sensitivity.

[0094] In an optional example, processing module 503 determines the bandwidth sensitivity for the nth adjustment based on the initial data transmission bandwidth, the data transmission bandwidth after the nth adjustment, and the buffer adjustment vector for the nth adjustment, expressed by the following expression:

[0095]

[0096] Among them, BW new BW represents the data transmission bandwidth after the nth adjustment. inital Let BS be the initial data transmission bandwidth, Δs be the buffer resource adjustment vector for the nth iteration, and BS be the buffer bandwidth. n Let n be the bandwidth sensitivity for the nth iteration.

[0097] In an optional example, processing module 503 is specifically used for:

[0098] Determine the average cache hit rate sensitivity based on the cache hit rate sensitivity after each cache resource adjustment;

[0099] And determine the average bandwidth sensitivity based on the bandwidth sensitivity after each cache resource adjustment;

[0100] After normalizing the average cache hit rate sensitivity and the average bandwidth sensitivity, the mean between the normalized average cache hit rate sensitivity and the normalized average bandwidth sensitivity is determined as the cache sensitivity corresponding to the first I / O data stream.

[0101] In an optional example, the second adjustment module 505 is specifically used to increase the cache resources of a preset number of I / O data streams with the highest cache sensitivity in the sorting according to a preset adjustment step size;

[0102] as well as,

[0103] Reduce cache resources by adjusting the preset number of I / O data streams with the lowest cache sensitivity according to the preset adjustment step size.

[0104] The description of the features of the cache partition dynamic allocation device provided in this application can be found in the relevant description of the cache partition dynamic allocation method, which will not be repeated here.

[0105] This application provides a dynamic cache partition allocation device. First, it obtains the initial cache hit rate and initial data transfer bandwidth of the I / O data stream after initialization. Then, it adjusts the cache resources corresponding to each I / O data stream multiple times according to a preset cache step size adjustment rule. After each adjustment, it obtains the cache hit rate and adjusted data transfer bandwidth, and then calculates the cache sensitivity of each I / O data stream based on these parameters. Next, it dynamically adjusts the cache partitions of each data stream based on its cache sensitivity, assigning higher priority to data streams that benefit most from the increased cache allocation. By dynamically adjusting partitions based on cache sensitivity, it effectively reduces performance degradation caused by cache pollution and resource contention. Furthermore, this application embodiment can accurately allocate cache resources based on the cache sensitivity of different data streams. It increases partitions for cache-sensitive data streams and reduces partitions for insensitive data streams with sufficient capacity, avoiding contention problems in traditional shared caches and unreasonable utilization of equally partitioned caches, thus enabling more efficient use of cache resources. Dynamically adjusting cache partitions allows each data stream to more stably obtain suitable cache resources, reducing performance fluctuations caused by insufficient or excessive cache usage, and improving system stability and reliability.

[0106] Embodiments of this application also provide an electronic device, such as... Figure 6 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described embodiments of the dynamic allocation method for cache partitioning.

[0107] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described embodiments of the dynamic allocation method for cache partitions, or to execute the steps in any of the above-described embodiments of the data reading method.

[0108] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0109] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the dynamic allocation method for cache partitions.

[0110] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data reading method embodiments.

[0111] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] The foregoing has provided a detailed description of a method, apparatus, device, and storage medium for dynamic allocation of cache partitions provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for dynamically allocating cache partitions, characterized in that, The method includes: After the cache resources corresponding to each I / O data stream of the multi-core dynamic hard disk have been initialized and configured, the initial cache hit rate and initial data transfer bandwidth of each I / O data stream during the operation are obtained. The cache resources corresponding to each I / O data stream are adjusted multiple times according to the preset cache step size adjustment rules, and the cache hit rate and data transmission bandwidth after each adjustment are obtained. Based on the initial cache hit rate and initial data transmission bandwidth corresponding to the first I / O data stream, the cache hit rate and data transmission bandwidth after each adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment, a cache sensitivity corresponding to the first I / O data stream is generated, where the first I / O data stream is any I / O data stream. After obtaining the cache sensitivity of each I / O data stream, sort the cache sensitivity of each I / O data stream. Based on the sorting results and the pre-configured cache resource adjustment rules, it is determined that the cache resources corresponding to some or all of the I / O data streams will be dynamically adjusted.

2. The method according to claim 1, characterized in that, The preset cache step size adjustment rule includes increasing or decreasing the cache resources corresponding to the I / O data stream by a multiple of the preset cache step size each time an adjustment is made.

3. The method according to claim 1, characterized in that, The step of generating a cache sensitivity corresponding to the first I / O data stream based on the initial cache hit rate and initial data transfer bandwidth corresponding to the first I / O data stream, the cache hit rate and data transfer bandwidth after each cache resource adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment, specifically includes: Based on the initial cache hit rate, the cache hit rate after the nth adjustment, and the cache adjustment vector of the nth adjustment, the cache hit rate sensitivity of the nth adjustment is determined. The cache adjustment vector of the nth adjustment includes the amount and direction of cache resource adjustment during the nth adjustment process. When the amount of cache resource adjustment is increased, the adjustment direction is positive, or when the amount of cache resource adjustment is decreased, the adjustment direction is negative, and n is a positive integer. The bandwidth sensitivity for the nth adjustment is determined based on the initial data transmission bandwidth, the data transmission bandwidth after the nth adjustment, and the buffer adjustment vector for the nth adjustment. Based on the cache hit rate sensitivity after each cache resource adjustment and the bandwidth sensitivity after each cache resource adjustment, a cache sensitivity corresponding to the first I / O data stream is generated.

4. The method according to claim 3, characterized in that, The cache hit rate sensitivity for the nth time is determined based on the initial cache hit rate, the cache hit rate after the nth adjustment, and the cache adjustment vector for the nth time, and is expressed by the following expression: Among them, HR new Let HR be the cache hit rate after the nth adjustment. inital Let Δs be the initial cache hit rate, and HS be the cache resource adjustment vector for the nth time. n Let be the cache hit rate sensitivity for the nth time.

5. The method according to claim 3, characterized in that, Based on the initial data transmission bandwidth, the data transmission bandwidth after the nth adjustment, and the buffer adjustment vector after the nth adjustment, the bandwidth sensitivity for the nth adjustment is determined, expressed by the following expression: Among them, BW new BW is the data transmission bandwidth after the nth adjustment. inital Let BS be the initial data transmission bandwidth, Δs be the buffer resource adjustment vector for the nth time, and BS be the buffer resource adjustment vector for the nth time. n The bandwidth sensitivity is given for the nth iteration.

6. The method according to any one of claims 3-5, characterized in that, The step of generating a cache sensitivity corresponding to the first I / O data stream based on the cache hit rate sensitivity and the bandwidth sensitivity after each cache resource adjustment specifically includes: Based on the cache hit rate sensitivity after each cache resource adjustment, determine the average cache hit rate sensitivity. And determine the average bandwidth sensitivity based on the bandwidth sensitivity after each cache resource adjustment; After normalizing the average cache hit rate sensitivity and the average bandwidth sensitivity respectively, the mean between the normalized average cache hit rate sensitivity and the normalized average bandwidth sensitivity is determined as the cache sensitivity corresponding to the first I / O data stream.

7. The method according to claim 1 or 2, characterized in that, The step of dynamically adjusting the cache resources corresponding to some or all of the I / O data streams based on the sorting results and pre-configured cache resource adjustment rules specifically includes: Increase cache resources for the preset number of I / O data streams with the highest cache sensitivity in the sorting according to the preset adjustment step size; as well as, Reduce cache resources by adjusting the preset number of I / O data streams with the lowest cache sensitivity according to the preset step size.

8. A dynamic allocation device for cache partitions, characterized in that, include: The acquisition module is used to acquire the initial cache hit rate and initial data transfer bandwidth of each I / O data stream during operation after the cache resources corresponding to each I / O data stream of the multi-core dynamic hard disk have been initialized and configured. The first adjustment module is used to adjust the cache resources corresponding to each I / O data stream multiple times according to the preset cache step size adjustment rules; The acquisition module is also used to acquire the adjusted cache hit rate and the adjusted data transmission bandwidth for each adjustment. The processing module is used to generate a cache sensitivity corresponding to the first I / O data stream based on the initial cache hit rate and initial data transmission bandwidth corresponding to the first I / O data stream, the cache hit rate and data transmission bandwidth after each adjustment obtained after multiple cache resource adjustments of the first I / O data stream, and the cache resource adjustment vector of the first I / O data stream for each adjustment. The first I / O data stream can be any I / O data stream. The sorting module is used to sort the cache sensitivities of all I / O data streams after obtaining them. The second adjustment module is used to determine, based on the sorting results and pre-configured cache resource adjustment rules, to dynamically adjust the cache resources corresponding to some or all of the I / O data streams.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the cache partition dynamic allocation method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the cache partition dynamic allocation method as described in any one of claims 1 to 7.

Images