Cache management of multiple applications

Abstract

The present disclosure relates to cache management for multiple applications. A system determines, by a network device, a total capacity of a cache to be used by multiple applications, the cache comprising memory distributed across multiple compute nodes. The system allocates a cache partition for a respective application from the total capacity by: determining an expected cache capacity for a current first time interval; determining an updated target hit rate based at least on a measured quality of service (QoS) value for a previous second time interval; and adjusting the expected cache capacity for the current first time interval based on the updated target hit rate.

Description

Background Technology

[0001] Managing performance presents significant challenges when multiple applications share limited resources, such as a distributed global cache. Two key considerations are: (1) determining the optimal cache capacity for each application, and (2) balancing cache allocation across applications. While traditional caching algorithms such as Least Recently Used (LRU) and Least Frequently Used (LFU) are typically used for data eviction when the cache reaches its capacity, these algorithms fail to provide sufficient performance isolation between different applications and may inadvertently favor one workload at the expense of another. Attached Figure Description

[0002] Figure 1A The illustration depicts an environment that facilitates cache management for multiple applications according to one aspect of this application, the environment including multiple local capacity controllers and a global arbitrator controller.

[0003] Figure 1B The illustration shows a method based on one aspect of this application. Figure 1A The environment has a global cache space with allocated cache partitions.

[0004] Figure 2A An adaptive cache capacity controller according to one aspect of this application is described.

[0005] Figure 2B An adaptive cache capacity controller according to one aspect of this application is described, the adaptive cache capacity controller including a global arbitrator controller.

[0006] Figure 3 A flowchart according to one aspect of this application is presented, illustrating a method for facilitating cache management for multiple applications.

[0007] Figure 4A A flowchart according to one aspect of this application is presented, illustrating a method for facilitating cache management for multiple applications.

[0008] Figure 4B A flowchart according to one aspect of this application is presented, illustrating a method for facilitating cache management for multiple applications.

[0009] Figure 5 The illustration shows a computer system that facilitates cache management for multiple applications according to one aspect of this application.

[0010] Figure 6 The illustration shows a computer-readable medium that facilitates cache management for multiple applications according to one aspect of this application.

[0011] In these accompanying figures, similar reference numerals refer to the same figure elements. Detailed Implementation

[0012] This application provides a system that facilitates cache management for multiple applications by providing an adaptive cache capacity controller.

[0013] Managing performance presents significant challenges when multiple applications share limited resources, such as a distributed global cache. Two key considerations are: (1) determining the optimal cache capacity for each application, and (2) balancing cache allocation across applications. While traditional caching algorithms such as Least Recently Used (LRU) and Least Frequently Used (LFU) are typically used for data eviction when the cache reaches its capacity, these algorithms fail to provide sufficient performance isolation between different applications and may inadvertently favor one workload at the expense of another.

[0014] The described aspects provide a system that facilitates cache management for multiple applications by partitioning a global cache and allocating a portion of the total cache capacity to each application based on several factors, including, for example, the application's target Quality of Service (QoS) value over a certain time interval, measured QoS value, target hit rate, and measured hit rate. The described system can provide an adaptive cache capacity controller comprising two types of controllers: a local capacity controller and a global arbitrator controller.

[0015] Each application's Local Capacity Controller (LCC) can perform two functions simultaneously. First, the LCC for the corresponding application can adjust the requested cache capacity for that application based on the previously assigned capacity and the difference between the target hit rate and the measured hit rate. Second, the LCC can modify the target hit rate based on a previous value of the target hit rate and the difference between the target QoS value and the measured QoS value. This adjusted target hit rate can be used to calculate further cache capacity requests. The LCC can perform these two functions in a nested loop, where the first function can be performed within a first time interval, and the second function can be performed within a second time interval greater than the first time interval.

[0016] A Global Arbiter Controller (GAC) can allocate a portion of the total cache capacity (or “cache partition”) to multiple LCCs for their respective applications by evaluating the requested capacity of each application based on its minimum acceptable capacity, relative priority, and a weighting factor used to allocate any remaining capacity. In this disclosure, the terms “global arbiter controller” and “global cache controller” are used interchangeably.

[0017] Figure 1A The illustration depicts an environment 100 facilitating cache management for multiple applications according to one aspect of this application. This environment includes multiple local capacity controllers (LCCs) 102, 104, and 106, and a global arbiter controller (GAC) 108. In environment 100, each LCC can determine the desired cache capacity for its respective application (as described below regarding...). Figure 2A and Figure 2B As described, the desired cache capacity is sent as a requested cache capacity to the GAC, which can apply a set of rules to optimally allocate the actual cache capacity to the corresponding application in each LCC. For example, LCC 102 can send a requested cache capacity (c) to GAC 108. req1 )112; LCC 104 can send a request for cache capacity (c) to GAC 108. req2 )114; and LCC 106 can send a request for cache capacity (c) to GAC 108. req3 116. GAC 108 can receive these multiple cache capacity requests 112 to 116 and more cache capacity requests from other LCCs (not shown).

[0018] GAC 108 can adjust the request cache capacity of an application by applying a set of rules based on information or hints provided by or related to the application. For example, GAC 108 can apply a first rule and allocate cache partitions (i.e., to its corresponding LCC) to the application based on the smaller of the application's expected cache capacity and its acceptable minimum cache capacity. Next, GAC 108 can apply a second rule and allocate cache partitions to the application based on the expected cache capacity sorted by the priority assigned to the application and the remaining cache capacity to be allocated. That is, if there is any remaining cache capacity after applying the first rule, GAC 108 can allocate additional cache capacity to these applications based on their respective priorities. Next, GAC 108 can apply a third rule and allocate cache partitions to these applications based on the weights assigned to them. That is, if there is any remaining cache capacity after applying the first and second rules, GAC 108 can allocate additional cache capacity to these applications based on their respective weights.

[0019] Therefore, as depicted in environment 100, GAC 108 can decide to adjust the request cache capacities 112, 114, and 116 from LCCs 102, 104, and 106, respectively, after applying the rules. GAC 108 can adjust the desired cache capacity by determining the optimal allocated cache capacity for each LCC. For example, GAC 108 can determine that allocated cache capacity (c1) 122 should be assigned to the cache partition of the corresponding application associated with LCC 102; GAC 108 can determine that allocated cache capacity (c2) 124 should be assigned to the cache partition of the corresponding application associated with LCC 104; and GAC 108 can determine that allocated cache capacity (c3) 126 should be assigned to the cache partition of LCC 106. The following section discusses… Figure 2B The role of GAC 108 in the adaptive cache capacity controller is further described.

[0020] Figure 1B The illustration shows a method based on one aspect of this application. Figure 1A Figure 130 shows a global cache space 132 with allocated cache partitions 142, 144, and 146. Figure 130 illustrates how the system determines the optimal cache partition size for each application and how the system determines how to balance partition sizes among applications. The allocated cache partition (c1) 142 can correspond to... Figure 1A The allocated cache capacity is 122; the allocated cache partition (c2) is 144, which can correspond to Figure 1A The allocated cache capacity is 124; and the allocated cache partition (c3) 146 can correspond to Figure 1A The allocated cache capacity is 126. The global cache space 132 may include memory distributed across multiple compute nodes (e.g., in the dynamic random access memory (DRAM) of the compute nodes), and may also be distributed across dedicated memory storage devices or the memory of a memory server.

[0021] Figure 2A An adaptive cache capacity controller 200 according to one aspect of this application is depicted. The controller 200 may exist as a hardware or software component in a network device. The controller 200 may include two separate controllers 206 and 210 performing different functions. The functions performed by the two separate controllers 206 and 210 may also be performed by fewer or more components, modules, or entities associated with the network device. Controller 206 may be referred to as "internal controller" 206 because it performs tasks corresponding to an internal loop based on a first "cycle" (i.e., a first time interval), and controller 210 may be referred to as "external controller 210" because it performs tasks corresponding to an external loop based on a second "cycle" (i.e., a second time interval) that is longer than the first cycle.

[0022] The internal controller 206 can adjust the request cache capacity to adjust the cache hit rate to a target cache hit rate, while the external controller 210 can adjust the target hit rate based on the difference between the measured QoS value or metric and the target QoS value or metric (e.g., response time).

[0023] The cache hit rate for any given workload is a monotonically non-decreasing function of the cache size. In other words, if the cache size is increased for any workload, the hit rate for that workload can only remain unchanged or increase, which can be expressed as follows: Equation (1) in, It's the hit rate. It's the cache capacity. It is the currently assigned capacity, and It's a tiny change in capacity. Therefore, the system can use adaptive control techniques to adaptively adjust the cache capacity to achieve any target hit rate regardless of workload.

[0024] The described aspects can be operated on a periodic or time-interval basis, and each application can use the total cache (i.e., as mentioned above regarding...). Figure 1B This refers to a known portion of the global cache space (132). Cache hit rate can be monitored as a common metric to determine whether more or less cache capacity is being provided to the corresponding application. Controlling cache hit rate is more intuitive because it is more directly related to cache capacity. For each application, the system can measure the corresponding application's performance over a given time period. The system observes the corresponding hit rate within the period. At the end of the period, the system can analyze the observed hit rate or other metrics, and the system can subsequently adjust the next time period. The allocation capacity. Assume the following variables: In the cycle The cache capacity internally allocated to the corresponding application. In the cycle Observed capacity usage during the period In the cycle The cache hit rate measured during the period, and It is the QoS value (e.g., average response time) measured during period j.

[0025] Cache hit rate and cache capacity It can exhibit a non-linear bimodal relationship. For example, the system can use the following model to approximate the cache hit rate. : Equation (2) in, It is the size of the working set, which is represented in the period. The maximum amount of data accessed during a given workload. (When the working set size...) Greater than cache capacity At that time, the cache was in an under-configured state, and the cache hit rate was low. With cache capacity Proportional (as shown in the top row of equation (2)). However, when the cache capacity Larger than the working set size At this time, the cache is over-configured, so the cache is large enough to hold the entire active working set, thereby improving the cache hit rate. The value is 1 (i.e., 100%) (as shown in the bottom row of equation (2)). Therefore, the described aspects provide a feedback control system that can adaptively adjust the cache capacity of a single application based on the nonlinear bimodal relationship between cache capacity and cache hit rate.

[0026] exist Figure 2A In this context, application 202 can run and output two values. The first value can be a measured QoS value. (220), and this first value can be used by the external controller 210, as described below. The second value can be the measured hit rate. (222), and this second value can be used by the internal controller 206 to determine the expected cache capacity for the current first time interval. As shown below: Equation (3) Equation (4) Equation (5) In equation (3), Indicates hit rate error, or target hit rate within the previous second time interval. The measured hit rate during the first time interval was compared with the previous one. The "first" difference between them is shown in equation (5). Target hit rate The default value can be set when application 202 initially runs. Application 202 can run for several initial time intervals. This is to allow the application to stabilize and output accurate measured QoS values ​​(220) and measured hit rates (222).

[0027] The first difference can be calculated. (Function 204), thus obtaining the hit rate error (224). The internal controller 206 can adjust according to the first gain parameter. Scale the first difference. This is a gain parameter, which can be based on the measured buffer capacity used in the previous interval. Adaptively self-adjusting. According to one aspect of this application, the adaptive gain parameter... This can be based on the non-linear bimodal relationship between cache hit rate and cache capacity. When cache capacity is insufficient (i.e., cache usage...) equal to the assigned cache capacity When the cache hit rate is limited by capacity (as shown in the top row of equation (4)), the internal controller 206 may need to aggressively allocate more capacity to quickly converge to the target hit rate. However, when the cache capacity is in a state of over-configuration (i.e., cache usage is excessive), the cache hit rate is limited by capacity. Less than the assigned cache capacity As shown in the bottom row of equation (4), the controller may expect to reduce capacity slowly to avoid exceeding the target, as exceeding the target may lead to serious QoS violations (e.g., longer response times).

[0028] variable and The adaptive gain, which is based on the measured hit rate during the previous first time interval, can be explained in the bottom and top rows of equation (4), respectively. With the identified cache capacity The dynamic value of the nonlinear bimodal relationship between them.

[0029] Therefore, the internal controller 206 can base its hit rate on the measured hit rate of the previous first time interval. (222) and the target hit rate in the previous second time interval (244) to adjust its expected cache capacity (226), that is, by executing function 204 to determine the hit rate error of the previous first time interval. (224), and the cache capacity assigned for the previous first time interval. According to the first gain parameter First difference of scaling (224) Perform a summation calculation. This expected (or requested) cache capacity can be assigned to application 202 as an allocated cache partition in its global cache space. Alternatively, the requested cache capacity can be... Submitted to the global arbitrator controller (e.g.) Figure 1AIn GAC 108, the global arbitrator controller can apply a set of rules and adjust the request cache capacity of application 202, as described below. Figure 2B As described.

[0030] In all the aspects described, the external controller 210 can achieve target hit rate based on the target QoS value. Automatic adjustment. Application 202 can adjust in the second time interval. Internal operation to output measured QoS values (220). Typically, the second time interval Compared to the first time interval Larger quantities, such as pre-ordered quantities or ratios. That is, application 202 can be completed in a shorter period. Run and output the measurement hit rate within that shorter period (e.g., 10 seconds). (222), and also outputs a longer period. Measured QoS values ​​within (e.g., 1 minute) (220). This allows the external controller 210 to adjust the cache hit rate for the current second time interval. As shown below: Equation (6) Equation (7) In equation (6), Error indicating QoS value, or target QoS value QoS values ​​measured during the previous second time interval The "second" difference between them is shown in equation (7).

[0031] The second difference can be calculated. (Function 208), thus obtaining the QoS value error (232). The external controller 210 can adjust the second gain parameter. Scaling this second difference. In all aspects described using response time or latency as a QoS metric, It is a fixed-gain parameter based on the monotonically non-decreasing relationship between target hit rate and measured QoS value. When response time or latency is used as a QoS metric, response time can be inversely proportional to target hit rate, which leads to a decrease in target hit rate. The target hit rate in the second time interval prior to this. With the second gain parameter The second difference of scaling The difference between them (rather than the sum as shown in equation (3)) is shown in equation (6).

[0032] In some respects, when using different QoS metrics that are proportional to the target hit rate, the target hit rate... It could be the target hit rate in the second previous time interval. With the second gain parameter The second difference of scaling The sum (similar to the sum in equation (3)). Furthermore, although the second gain parameter... In equation (6), it is depicted as a fixed, non-dynamic value based on a monotonically non-decreasing relationship between the target hit rate and the measured QoS value. However, other QoS metrics that are related to the target hit rate can be used, which requires the use of adaptive gain parameters. .

[0033] Therefore, the external controller 210 can measure the QoS value based on the previous second time interval. (220) and target QoS value (230) to update the target hit rate for the current second time interval. (234), that is, the QoS value error of the previous second time interval is determined by executing function 208. (232), and calculate the target hit rate for the previous second time interval. With the second gain parameter The second difference of scaling The difference between (232). The calculated hit rate of the current second time interval after a one-step delay (212). (234) can be used as the target hit rate for the previous second time interval. (244). Target hit rate (244) can be output and fed into a calculation performed by internal controller 206 (at function 204) to obtain the hit rate error (224, as described above with respect to equation (5)), thereby adjusting the request cache capacity by adjusting the cache hit rate to the updated target hit rate.

[0034] Figure 2B An adaptive cache capacity controller 250 according to one aspect of this application is depicted, the adaptive cache capacity controller including a global arbiter controller 262. Similar to... Figure 2A Controllers 200 and 250 can exist as hardware or software components in the network device. Controller 250 may include two separate controllers (internal controller 256 and external controller 260) that perform different functions (similar to...). Figure 2A(Internal controller 206 and external controller 210). Controller 250 may also include a global arbitrator controller 262. The functions performed by the two separate controllers 256 and 260 and GAC 262 may also be performed by fewer or more components, modules or entities associated with the network device.

[0035] Similar to Figure 2A Applications 202 and 252 can run and output two values. The first value can be a measured QoS value. (270), and the second value can be the measured hit rate. (272). These values ​​can be used as described below. Similar to those described by... Figure 2A The functions performed by the internal controller 206 and the external controller 210, wherein the internal controller 256 can measure the hit rate based on the previous first time interval. (272) and the target hit rate in the previous second time interval (294) to adjust its expected cache capacity (276), that is, by executing function 254 to determine the hit rate error of the previous first time interval. (274), and the cache capacity assigned for the previous first time interval. According to the first gain parameter First difference of scaling (274) Perform a summation calculation as described above with respect to equations (3) to (5). Furthermore, the external controller 260 can perform the calculation based on the measured QoS value from the previous second time interval. (270) and target QoS value (280) to update the current second time interval. Target hit rate (284), that is, by performing function 258, the QoS value error of the previous second time interval is determined. (282), and calculate the target hit rate for the previous second time interval. With the second gain parameter The second difference of scaling The difference between (282) and (262) is as described above with respect to equations (6) to (7). The calculated hit rate for the current second time interval after a one-step delay (262) is... (284) can be used as the target hit rate for the previous second time interval. (294). Target hit rate (294) can be output and fed into a calculation performed by the internal controller 256 (at function 254), thereby adjusting the request cache capacity by adjusting the cache hit rate to the updated target hit rate.

[0036] exist Figure 2B In the diagram, controller 250 also describes a GAC ​​262, which can cache request capacity from multiple LCCs (e.g., request cache capacity from internal controller 256). (276), or Figure 1A The request cache capacity (any one of 112, 114, and 116) is taken as input, and the request cache capacity from each LCC is adjusted by applying a set of rules based on information associated with the corresponding application. GAC 262 can obtain this information from various entities, including, for example, the application itself, a central entity, another network device, or system configuration. This information may include, for example: the minimum acceptable cache capacity for the corresponding application; the priority assigned to the corresponding application; and the weight assigned to the corresponding application.

[0037] As mentioned above Figure 1A As described, GAC 262 can allocate cache partitions for each application and its corresponding LCC by applying the following rules in sequence, based on the smaller of the expected cache size and the minimum cache size; the priority assigned to the corresponding application and the remaining global cache space to be allocated; and the weight assigned to the corresponding application. For example, in Figure 2B In GAC 262, a set of ordered rules can be applied, and the request cache capacity can be adjusted. (276) Adjusted to the adjusted capacity allocation (278). The system can then adjust the capacity. (278) Assigned to application 252 as its period or time interval The cache partition within.

[0038] Therefore, the LCC's internal controller can adjust the application's requested cache capacity based on the previously assigned capacity and the difference between the target hit rate and the measured hit rate (as shown in equations (3) to (5)). Simultaneously, the LCC's external controller can modify the target hit rate based on a previous value of the target hit rate and the difference between the target QoS value and the measured QoS value (as shown in equations (6) to (7)). This adjusted target hit rate can be used to calculate further cache capacity requests. The LCC (i.e., through the internal and external controllers) can execute these two functions in a nested loop, where the first function can be executed within a first time interval, and the second function can be executed within a second time interval greater than the first time interval. Subsequently, LCCs associated with multiple applications can send their requested cache capacity to the GAC, which can adjust the cache partition allocation for each LCC based on the aforementioned set of rules. The operation and interaction between the LCC and the GAC can produce a system that determines the optimal cache capacity for each application and also balances cache partitions between applications, thereby achieving efficient performance isolation between applications.

[0039] Figure 3 A flowchart 300 according to one aspect of this application is presented, illustrating a method for facilitating cache management for multiple applications. During operation, the system determines, via a network device, the total capacity of a cache to be used by multiple applications, which includes memory distributed across multiple compute nodes (operation 302). The cache may be referred to as a "global cache," which is distributed across a network including compute nodes and dedicated storage servers, rather than being limited to a single machine. For example, Figure 1B The global cache space 132 can include a combination of DRAM from compute nodes and dedicated storage servers, which allows cached data to be accessed and shared by all instances of the application or system.

[0040] The system allocates cache partitions for the corresponding applications from the total capacity (Operation 304). The system may include a local capacity controller that performs both functions. For example, Figure 2A The adaptive cache controller 200 may include an internal controller 206 that adjusts the requested cache capacity based on a previously assigned cache capacity, a target hit rate, and a measured hit rate, as described above regarding function 204. The external controller 210 may modify the target hit rate based on a previous value of the target hit rate, a target QoS value, and a measured QoS value, as described above regarding function 208. Each controller may perform the functions described above regarding... Figure 2A and Figure 2B The described functionality.

[0041] Allocating cache partitions for the corresponding application may include operation 310, which includes operations 312, 314, and 316. The system determines the desired cache capacity within the current first time interval (operation 312). For example, Figure 2A The internal controller 206 can determine the request cache capacity based at least on the measured hit rate, the target hit rate, and the hit rate error (i.e., a first difference between the target hit rate and the measured hit rate). (226). The internal controller 206 can also be based on the first gain parameter. And the above about Figure 2A The calculations described by equations (3) to (5) determine the request cache capacity of 226.

[0042] The system determines the updated target hit rate based at least on the measured Quality of Service (QoS) values ​​from the previous second time interval (Operation 314). Figure 2A In this context, the external controller 210 can determine the target hit rate based at least on the measured QoS value, the target QoS value, and the QoS value error (i.e., a second difference between the target QoS value and the measured QoS value). (234). The external controller 210 can also be based on the second gain parameter. And the above about Figure 2A The calculations described in equations (6) to (7) are used to determine the target hit rate 234.

[0043] The system adjusts the expected cache capacity for the current first time interval based on the updated target hit rate (operation 316). For example, internal controller 206 can receive an updated target hit rate 234 as input from external controller 210 (this updated target hit rate is calculated based on function 208 performed by external controller 210). Internal controller 206 can perform function 204, which involves calculating the error of the hit rate 234 (i.e., the first difference), and subsequently using this difference to determine the request cache capacity, as described above regarding... Figure 2A And as described by equations (3) to (5). The system can be used in the first time interval ( The internal controller 206 continues to perform its functions within the second time interval ( ) and in the second time interval ( The external controller 210 continues to perform its functions, as described above. Figure 1A and Figure 2A As described. The operation described in flowchart 300 returns.

[0044] Figure 4AA flowchart 400 according to one aspect of this application is presented, illustrating a method for facilitating cache management for multiple applications. Flowchart 400 depicts operations performed by the internal controller of the LCC as part of an internal loop (left side, including decisions 410-412 and operations 414-420), and operations performed by the external controller of the LCC as part of an external loop (right side, including decisions 430-432 and operations 434-440). The operations performed by the internal controller as depicted on the left side of flowchart 400 can be based on a first time interval. Furthermore, the operations performed by the external controller, as depicted on the right side of flowchart 400, can be based on a second time interval. The second time interval is longer than the first time interval. Therefore, operations executed by the internal controller and the external controller can be performed simultaneously, as described above. Figure 2A and Figure 2B The loop described.

[0045] If the system does not receive the measurement hit rate from the application (Decision 410), the operation waits to receive the measurement hit rate. If the system does receive the measurement hit rate from the application (Decision 410), and if the first time interval has not yet been reached... (Decision 412) The operation waits until the first time interval is reached. If the system does receive a measurement hit rate from the application (Decision 410), and if the first time interval has been reached... (Decision 412), then the system identifies the cache capacity as previously allocated in the first time interval (Operation 414). For example, Figure 2A The internal controller 206 can recognize the information described above regarding equation (3). .

[0046] The system determines a first difference between the target hit rate and the measured hit rate cached during the previous first time interval (operation 416). This first difference can be a hit rate error, as described above regarding equation (5) and... Figure 2A Element 224 describes this.

[0047] The system selects a first gain parameter associated with the first difference (operation 418). The first gain parameter can be as described above. Figure 2A The adaptive self-tuning parameters described by equation (4) This adaptive self-tuning parameter is further based on two value( and These two values ​​are selected based on the relationship between the assigned capacity and the utilized capacity in the previous first time interval.

[0048] The system calculates the expected cache capacity for the current first time interval based on the sum of the following: the identified cache capacity; and a first difference scaled by a first gain parameter (operation 420). (See above regarding...) Figure 2A As described, the internal controller 206 can calculate the current first time interval based on the sum of the following items. The requested cache capacity (226): the identified cache capacity (assigned during the previous first time interval); and the hit rate error scaled by the first gain parameter, as described above with respect to equation (3). Operation in Figure 4B Continue at label A.

[0049] If the system does not receive a measured QoS value from the application (Decision 430), the operation waits to receive the measured QoS value. If the system does receive a measured QoS value from the application (Decision 410), and if the second time interval has not yet been reached... (Decision 432) The operation waits until the second time interval is reached. If the system does receive a measured QoS value from the application (Decision 430), and if the second time interval has been reached... (Decision 432) Then the system identifies the target hit rate of the previous second time interval (Operation 434). For example, Figure 2A The external controller 210 can recognize the information described above regarding equation (6). .

[0050] The system determines a second difference between measured QoS values ​​within a previous second time interval for the target QoS value (operation 436). This second difference can be an error in the QoS value, as described above regarding... Figure 2A And as described by equation (7).

[0051] The system selects a second gain parameter associated with the second difference (operation 438). The second gain parameter can be as described above. Figure 2A And the fixed parameters described by equation (6) In some respects, the second gain parameter It can be a dynamically changing value, similar to the adaptive first gain parameter. This depends on whether the relationship is non-linear, bimodal, or monotonic, etc.

[0052] The system calculates the updated target hit rate within the current second time interval based on a third difference between the following two items: the target hit rate of the previous second time interval; and a second difference scaled by the second gain parameter (operation 440). (See above regarding...) Figure 2A As described, the external controller 210 can calculate the updated target hit rate for the current second time interval based on the difference between the following two items. (234): Target hit rate in the previous second time interval; error with the QoS value scaled by the second gain parameter, as described above regarding Figure 2A And the operations described in equations (6) to (7). Figure 4B Continue at label A.

[0053] Figure 4B A flowchart 450 according to one aspect of this application is presented, illustrating a method for facilitating cache management across multiple applications. The system (e.g., a local cache controller (LCC)) sends a request for the desired cache capacity to the global arbitrator controller (GAC) (operation 452). For example, in Figure 1A In the middle, LCC 104 sends a request for cache capacity to GAC 108. (114). As another example, in Figure 2B In the middle, the internal controller 256 sends a request for cache capacity to the GAC 262. (276).

[0054] The system receives requests for desired cache capacity and other requests (Operation 454) from the LCCs of network devices associated with multiple applications via the GAC. For example, Figure 1A GAC 108 can receive requests for cache sizes of 112, 114, and 116 from LCCs 102, 104, and 106, respectively.

[0055] The system uses the GAC to adjust the expected cache capacity requests for the corresponding application by applying a set of rules (Operation 456). Figure 1A GAC 108 and Figure 2B GAC 262 can be applied to allocate global cache space among multiple applications. The rules can be an ordered or sequential set of rules and can be based on information provided by or associated with the respective application. As described above regarding GAC 262, this information may include, for example: the minimum cache size acceptable to the respective application; the priority assigned to the respective application; and the weight assigned to the respective application.

[0056] The system allocates (or assigns) the adjusted expected cache capacity to the corresponding application via the GAC (Operation 458). Figure 1A In this context, GAC 108 can respond to a request received from LCC 102. (112) and the adjusted cache capacity (122) Assigned to LCC 102. Additionally, in Figure 2B In GAC 262, a globally distributed cache space (e.g., Figure 1BThe adjusted capacity of the global cache space (132) (278) Assigned to application 252.

[0057] The system utilizes the allocated cache partition through the corresponding application (Operation 460). For example, Figure 2B Application 252 can be allocated adjusted capacity. (278), which allows application 252 to utilize the allocated cache partition in order to continue running. Operation returns.

[0058] Figure 5 The illustration depicts a computer system 500 that facilitates cache management for multiple applications according to one aspect of this application. The computer system 500 includes a processor 502, a memory 504, and a storage device 506. The memory 504 may include volatile memory (e.g., random access memory (RAM)) that serves as managed memory and can be used to store one or more memory pools. Furthermore, the computer system 500 may be coupled to peripheral I / O user devices 510 (e.g., a display device 511, a keyboard 512, and a pointing device 513). The storage device 506 includes a non-transitory computer-readable storage medium and stores an operating system 516, instructions 518, and data 530. The computer system 500 may include a processor 502, a memory 504, and a storage device 506. Figure 5 The entities or instructions shown are fewer or more entities or instructions.

[0059] When executed by computer system 500, instruction 518 can cause computer system 500 to perform the methods and / or processes described in this disclosure. Specifically, instruction 518 may include instruction 520 for determining, by a network device, the total capacity of a cache to be used by multiple applications, the cache including memory distributed across multiple computing nodes, as described above. Figure 1B Global cache space 132 and Figure 3 As described in operation 302.

[0060] Instruction 518 may include instruction 522 for allocating cache partitions for the corresponding applications from the total capacity, as described above regarding the allocation of cache partitions by the application. Figure 2A The functions performed by the internal controller 206 and the external controller 210 and Figure 3 Operations 304 and 310 are described.

[0061] Instructions 522 for allocating cache partitions for the corresponding application may include instructions 524 for determining the desired cache capacity within the current first time interval, as described above regarding the allocation of cache partitions for the application. Figure 2A The internal controller 206 and Figure 2BThe internal loop functions executed by the internal controller 256 in functions 204 and 254 are described above. Also mentioned above are equations (3) through (5) and Figure 3 Operation 312 describes the determination of the desired cache size.

[0062] Instruction 522 for allocating cache partitions for the corresponding application may further include instruction 526 for determining an updated target hit rate based at least on measured Quality of Service (QoS) values ​​over a previous second time interval, as described above regarding the... Figure 1A External controller 210 and Figure 2B The external controller 260 is described in terms of the external loop functions executed in functions 208 and 258, respectively. Equations (6) to (7) and... are also mentioned above. Figure 3 Operation 314 describes the determination of the updated target hit rate.

[0063] Instruction 522 for allocating cache partitions for the corresponding application may further include instruction 528 for adjusting the expected cache capacity within the current first time interval based on the updated target hit rate, as described above regarding the allocation of cache partitions for the corresponding application. Figure 2A Internal controller 206 / External controller 210 and Figure 2B The interaction between the two loops executed by the internal controller 256 / external controller 260 and the functions they perform. Figure 3 As described in operation 316.

[0064] Instruction 518 may include more than Figure 5 The instructions shown are further instructions. For example, instruction 518 may include instructions for performing the operations described above with respect to the following: Figure 1A and Figure 1B The environment; Figure 2A and Figure 2B Functions and communication; Figure 3 , Figure 4A and Figure 4B Operations within the flowchart; and Figure 6 The instructions in CRM 600.

[0065] Data 530 may include any data that is required as input or generated as output by the methods, operations, communications and / or processes described in this disclosure. Specifically, data 530 may store at least: the total cache capacity; indicators or identifiers of network devices or applications; identifiers of DRAM, compute nodes, or dedicated memory servers; indicators of applications and corresponding LCCs; indicators of allocated cache partitions or cache capacity; desired cache capacity; assigned cache capacity; the amount of cache capacity utilized or used; the current or previous first time interval; the current or previous second time interval; hit rate; target hit rate; measured hit rate; QoS value or metric; target QoS value or metric; measured QoS value or metric; updated target hit rate; adjusted desired cache capacity; a predetermined amount; cache capacity assigned for the previous first time interval; a first difference; a first gain parameter; a sum; the identified target hit rate for the previous second time interval; a second difference; a second gain parameter; an adaptive or dynamic gain parameter; a fixed gain parameter; a third difference; a sum; an indicator of a non-linear bimodal relationship; an indicator of a monotonic relationship; a fixed value; a dynamic value; an indicator of a non-linear relationship; a set of rules; a set of ordered rules; a set of sequential rules; minimum acceptable cache capacity; priority; and weights.

[0066] Figure 6 The illustration shows a computer-readable medium (CRM) 600 facilitating cache management for multiple applications according to one aspect of this application. The CRM 600 may be a non-transitory computer-readable medium or device storing instructions that, when executed by a computer or processor, cause the computer or processor to perform a method. The CRM 600 may store instructions 610 for determining (i.e., for determining) the total capacity of a cache to be used by multiple applications, the cache including memory distributed across multiple computing nodes, as described above. Figure 1B Global cache space 132 and Figure 3 As described in operation 302.

[0067] The CRM 600 can store instructions 620 for allocating cache partitions for the corresponding applications from the total capacity, as mentioned above regarding... Figure 2A The functions performed by the internal controller 206 and the external controller 210 and Figure 3 Operations 304 and 310 are described.

[0068] In CRM 600, the instruction 620 for allocating cache partitions for the corresponding application may include an instruction 630 for determining the expected cache capacity within the current first time interval, as described above regarding the... Figure 2A The internal controller 206 and Figure 2BThe internal loop functions executed by the internal controller 256 in functions 204 and 254 are described above. Also mentioned above are equations (3) through (5) and Figure 3 Operation 312 describes the determination of the desired cache size.

[0069] In CRM 600, the instruction 620 for allocating cache partitions for the corresponding application may include instruction 640 for determining an updated target hit rate based at least on measured Quality of Service (QoS) values ​​over a previous second time interval, as described above regarding the... Figure 1A External controller 210 and Figure 2B The external controller 260 is described in terms of the external loop functions executed in functions 208 and 258, respectively. Equations (6) to (7) and... are also mentioned above. Figure 3 Operation 314 describes the determination of the updated target hit rate.

[0070] In CRM 600, the instruction 620 for allocating cache partitions for the corresponding application may include an instruction 650 for adjusting the expected cache capacity within the current first time interval based on the updated target hit rate, as described above regarding the... Figure 2A Internal controller 206 / External controller 210 and Figure 2B The interaction between the two loops of the internal controller 256 / external controller 260 and the functions they perform. Figure 3 As described in operation 316.

[0071] CRM 600 can include more Figure 6 The instructions shown are further instructions. For example, the CRM 600 can also store instructions for performing the operations described above with respect to the following: Figure 1A and Figure 1B The environment; Figure 2A and Figure 2B Functions and communication; Figure 3 , Figure 4A and Figure 4B Operations within the flowchart; and Figure 5 The instructions of computer system 500.

[0072] The term "network device" refers to any device, component, or computing entity that can provide a communication channel for data packets sent from a "processing node" or "endpoint node." A processing node or endpoint node can refer to a device, component, or hardware component that can operate as a source or destination of data, including, for example, control packets or data packets. A network device can include ingress network devices, intermediate or intermediate structure network devices, or egress or endpoint network devices. An example of a network device can be a switch. A processing node or endpoint node can include an ingress node (which is the endpoint from which data is returned upon request) or an egress node (which is the endpoint from which data is sent upon request).

[0073] In general, the disclosed aspects provide a computing system, method, and computer-readable medium that facilitate cache management for multiple applications. The system determines, via a network device, the total capacity of a cache to be used by multiple applications, the cache comprising memory distributed across multiple computing nodes. The system allocates cache partitions for the respective applications from the total capacity by: determining the desired cache capacity for the current first time interval; determining an updated target hit rate based at least on measured Quality of Service (QoS) values ​​from a previous second time interval; and adjusting the desired cache capacity for the current first time interval based on the updated target hit rate.

[0074] In variations of this, the cache further includes at least one of the following: dynamic random access memory (DRAM) of multiple compute nodes; or memory of a dedicated memory server.

[0075] In a further variant of this, the system dynamically adjusts the expected cache capacity for the subsequent first time interval based on the previously assigned cache capacity and the updated target hit rate.

[0076] In a further variant, the second time interval is a predetermined amount larger than the first time interval.

[0077] In a further variant, the system determines the expected cache capacity for the corresponding application within the current first time interval by performing the following operations: The system identifies the cache capacity assigned in the previous first time interval. The system determines a first difference between the updated target hit rate and the measured hit rate in the cache during the previous first time interval. The system selects a first gain parameter associated with the first difference. The system calculates the expected cache capacity within the current first time interval based on the sum of the identified cache capacity and the first difference scaled by the first gain parameter.

[0078] In a further variant, the system determines the updated target hit rate for the corresponding application within the current second time interval by performing the following operations: The system identifies the target hit rate of the previous second time interval. The system determines a second difference between the target QoS value and the measured QoS value within the previous second time interval. The system selects a second gain parameter associated with the second difference. The system calculates the updated target hit rate within the current second time interval based on a third difference between: the target hit rate of the previous second time interval; and the second difference scaled by the second gain parameter.

[0079] In a further variant, the first gain parameter includes a dynamic value based on a non-linear bimodal relationship between the measured hit rate and the identified cache capacity based on a previous first time interval. The second gain parameter includes at least one of the following: a fixed value based on a monotonic relationship between the updated target hit rate and the measured QoS value based on the corresponding application; or a dynamic value based on a non-linear relationship between the updated target hit rate and the measured QoS value based on the corresponding application.

[0080] In a further variant, the system allocates cache partitions to the respective application by adjusting the expected cache capacity of the application (e.g., by applying a set of rules). These rules include: allocating cache partitions based on the smaller of the application's expected cache capacity and its minimum acceptable cache capacity; allocating cache partitions based on the expected cache capacity sorted by the priority assigned to the application and the remaining cache capacity to be allocated; and allocating cache partitions based on the weight assigned to the application.

[0081] In a further variant, the system receives requests for desired cache capacity from local cache controllers of network devices associated with multiple applications via a global cache controller associated with the network device. The system then adjusts the desired cache capacity for the respective applications by applying this set of rules through the global cache controller.

[0082] On the other hand, a computing system includes processing resources and storage devices for storing instructions that, when executed by the processing resources, cause the processing resources to perform a method. The method includes determining, by a network device, the total capacity of a cache to be used by multiple applications, the cache comprising memory distributed across multiple computing nodes. The method further includes allocating cache partitions from the total capacity for corresponding applications. Allocating cache partitions for corresponding applications includes: determining a desired cache capacity within a current first time interval; determining an updated target hit rate based at least on measured Quality of Service (QoS) values ​​from a previous second time interval; and adjusting the desired cache capacity within the current first time interval based on the updated target hit rate. The computing system may also include additional instructions related to: Figure 1A and Figure 1B The environment; Figure 2A and Figure 2B Functions and communication; Figure 3 , Figure 4A and Figure 4B Operations within the flowchart; and Figure 6 The instructions in CRM 600.

[0083] On the other hand, a non-transitory computer-readable storage medium (or CRM) stores instructions that, when executed by a processing resource or computer, cause the processing resource or computer to perform a method. The method includes determining, by a network device, the total capacity of a cache to be used by multiple applications, the cache comprising memory distributed across multiple computing nodes. The method further includes allocating cache partitions from the total capacity for corresponding applications. Allocating cache partitions for corresponding applications includes: determining a desired cache capacity within a current first time interval; determining an updated target hit rate based at least on measured Quality of Service (QoS) values ​​from a previous second time interval; and adjusting the desired cache capacity within the current first time interval based on the updated target hit rate. The CRM may also store instructions for performing the operations described above with respect to the following: Figure 1A and Figure 1B The environment; Figure 2A and Figure 2B Functions and communication; Figure 3 , Figure 4A and Figure 4B Operations within the flowchart; and Figure 5 The instructions of computer system 500.

[0084] The foregoing description is presented to enable any person skilled in the art to make and use the aspects and examples, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects and applications without departing from the spirit and scope of this disclosure. Therefore, the aspects described herein are not limited to those shown, but are intended to be consistent with the maximum scope of the principles and features disclosed herein.

[0085] Furthermore, the foregoing descriptions of the various aspects have been presented solely for illustrative and descriptive purposes. These descriptions are not intended to be exhaustive or to limit the aspects described herein to the disclosed forms. Accordingly, many modifications and variations will be apparent to those skilled in the art. Additionally, the foregoing disclosure is not intended to limit the aspects described herein. The scope of the aspects described herein is defined by the appended claims.

Claims

1. A method for performing computation, comprising: The total capacity of the cache to be used by multiple applications is determined by the network device, and the cache includes memory distributed across multiple computing nodes; as well as Allocating cache partitions for the corresponding applications from the total capacity, wherein allocating the cache partitions for the corresponding applications includes: Determine the expected cache capacity within the current first time interval; The updated target hit rate is determined at least based on the measured Quality of Service (QoS) values ​​within the previous second time interval; and The expected cache capacity within the current first time interval is adjusted based on the updated target hit rate.

2. The computational method of claim 1, wherein the cache further comprises at least one of the following: The dynamic random access memory (DRAM) of the multiple computing nodes; or The storage of a dedicated storage server.

3. The calculation method as described in claim 1, further comprising: The expected cache capacity in subsequent first time intervals is dynamically adjusted based on the previously assigned cache capacity and the updated target hit rate.

4. The calculation method as described in claim 1, wherein, The second time interval is a predetermined amount larger than the first time interval.

5. The calculation method as described in claim 1, wherein, Determining the expected cache capacity of the corresponding application within the current first time interval includes: Identify the cache capacity assigned in the previous first time interval; Determine a first difference between the updated target hit rate and the measured hit rate in the cache during the previous first time interval; Select a first gain parameter associated with the first difference; and The expected cache capacity for the current first time interval is calculated based on the sum of the following: The identified cache capacity; and The first difference scaled by the first gain parameter.

6. The calculation method as described in claim 5, wherein, Determining the target hit rate of the update of the corresponding application within the current second time interval includes: Identify the target hit rate of the previous second time interval; Determine a second difference between the target QoS value and the measured QoS value within the previous second time interval; Select a second gain parameter associated with the second difference; and The updated target hit rate within the current second time interval is calculated based on a third difference between the following two items: The target hit rate during the previous second time interval; and The second difference, scaled by the second gain parameter.

7. The calculation method as described in claim 6, in, The first gain parameter includes a dynamic value based on the nonlinear bimodal relationship between the measured hit rate and the identified cache capacity over the previous first time interval; and The second gain parameter includes at least one of the following: A fixed value based on the monotonic relationship between the updated target hit rate and the measured QoS value of the corresponding application; or The dynamic value is based on the nonlinear relationship between the updated target hit rate and the measured QoS value of the corresponding application.

8. The calculation method as described in claim 1, wherein, Allocating the cache partition for the corresponding application further includes: The desired cache capacity of the corresponding application is adjusted by applying a set of rules, which include: The cache partition is allocated based on the smaller of the expected cache capacity and the minimum acceptable cache capacity of the corresponding application; The cache partitions are allocated based on the expected cache capacity sorted by priority assigned to the respective applications and the remaining cache capacity to be allocated; and The cache partitions are allocated based on the weights assigned to the respective applications.

9. The calculation method as described in claim 8, further comprising: The global cache controller associated with the network device receives a request for the desired cache capacity from the local cache controller of the network device associated with the plurality of applications; as well as The global cache controller adjusts the desired cache capacity of the corresponding application by applying the set of rules.

10. A computing system, comprising: Processing resources; as well as A storage device storing instructions, which, when executed by the processing resource, cause the processing resource to execute a method, the method comprising: The total capacity of a cache to be used by multiple applications is determined by the network device, the cache including memory distributed across multiple compute nodes; and Allocating cache partitions for the corresponding applications from the total capacity, wherein allocating the cache partitions for the corresponding applications includes: Determine the expected cache capacity within the current first time interval; The updated target hit rate is determined at least based on the measured Quality of Service (QoS) values ​​within the previous second time interval; and The expected cache capacity within the current first time interval is adjusted based on the updated target hit rate.

11. The computing system of claim 10, wherein the method further comprises: The expected cache capacity in subsequent first time intervals is dynamically adjusted based on the previously assigned cache capacity and the updated target hit rate. The second time interval is a predetermined amount larger than the first time interval.

12. The computing system of claim 10, wherein, Determining the expected cache capacity of the corresponding application within the current first time interval includes: Identify the cache capacity assigned in the previous first time interval; Determine a first difference between the updated target hit rate and the measured hit rate in the cache during the previous first time interval; Select a first gain parameter associated with the first difference; and The expected cache capacity for the current first time interval is calculated based on the sum of the following: The identified cache capacity; and The result of scaling the first difference by the first gain parameter.

13. The computing system of claim 12, wherein, Determining the target hit rate of the update of the corresponding application within the current second time interval includes: Identify the target hit rate of the previous second time interval; Determine a second difference between the target QoS value and the measured QoS value within the previous second time interval; Select a second gain parameter associated with the second difference; and The updated target hit rate within the current second time interval is calculated based on a third difference between the following two items: The target hit rate during the previous second time interval; and The result of scaling the second difference by the second gain parameter.

14. The computing system as described in claim 13, in, The first gain parameter includes a dynamic value based on the nonlinear bimodal relationship between the measured hit rate and the identified cache capacity over the previous first time interval; and The second gain parameter includes at least one of the following: A fixed value based on the monotonic relationship between the updated target hit rate and the measured QoS value of the corresponding application; or The dynamic value is based on the nonlinear relationship between the updated target hit rate and the measured QoS value of the corresponding application.

15. The computing system of claim 10, wherein the method further comprises: The desired cache capacity of the corresponding application is adjusted by applying a set of rules. The set of rules includes: The cache partition is allocated based on the smaller of the expected cache capacity and the minimum acceptable cache capacity of the corresponding application; The cache partitions are allocated based on the expected cache capacity sorted by priority assigned to the respective applications and the remaining cache capacity to be allocated; and The cache partitions are allocated based on the weights assigned to the respective applications.

16. The computing system of claim 15, wherein the method further comprises: The global cache controller associated with the network device receives a request for the desired cache capacity from the local cache controller of the network device associated with the plurality of applications; as well as The global cache controller adjusts the desired cache capacity of the corresponding application by applying the set of rules.

17. A non-transitory computer-readable medium storing instructions, said instructions causing said processing resource to perform a method when executed by said processing resource, said method comprising: The total capacity of the cache to be used by multiple applications is determined by the network device, and the cache includes memory distributed across multiple computing nodes; as well as Allocating cache partitions for the corresponding applications from the total capacity, wherein allocating the cache partitions for the corresponding applications includes: Determine the expected cache capacity within the current first time interval; The updated target hit rate is determined at least based on the measured Quality of Service (QoS) values ​​within the previous second time interval; and The expected cache capacity within the current first time interval is adjusted based on the updated target hit rate.

18. The non-transitory computer-readable medium of claim 17, wherein the method further comprises: The expected cache capacity in subsequent first time intervals is dynamically adjusted based on the previously assigned cache capacity and the updated target hit rate.

19. The non-transitory computer-readable medium of claim 17, wherein, Determining the expected cache capacity of the corresponding application within the current first time interval includes: Identify the cache capacity assigned in the previous first time interval; Determine a first difference between the updated target hit rate and the measured hit rate in the cache during the previous first time interval; Select a first gain parameter associated with the first difference; and The expected cache capacity for the current first time interval is calculated based on the sum of the following: The identified cache capacity; and The result of scaling the first difference by the first gain parameter.

20. The non-transitory computer-readable medium of claim 19, wherein, Determining the target hit rate of the update of the corresponding application within the current second time interval includes: Identify the target hit rate of the previous second time interval; Determine a second difference between the target QoS value and the measured QoS value within the previous second time interval; Select a second gain parameter associated with the second difference; and The updated target hit rate within the current second time interval is calculated based on a third difference between the following two items: The target hit rate during the previous second time interval; and The result of scaling the second difference by the second gain parameter.

Images