A distributed cache system to prevent cache stampede

By introducing a promise protocol and a current limiting mechanism of time hash function in the distributed cache system, the cache pedaling problem in the cache miss is solved, and the system stability and performance guarantee in huge concurrent scenarios are achieved.

CN114116796BActive Publication Date: 2024-12-03上海沄熹科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111287708.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-02
Publication Date
2024-12-03
Estimated Expiration
2041-11-02

AI Technical Summary

Technical Problem

Distributed cache systems are prone to cache trampling when cache is not pre-stored or suddenly fail, resulting in excessive database pressure or even downtime.

Method used

A distributed cache system is designed to prevent cache trampling. By introducing protocol parsers, cache processors, key counters, control configuration items, current limiters and protocol generators, the promised protocol and time hash functions are used to concurrency in huge amounts. Active current limit is carried out in the scenario to avoid huge requests in a short time when a cache is missed.

Benefits of technology

It effectively avoids cache trampling, controls the time and number of request retry times, avoids excessive database pressure and ensures stable system operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114116796B_ABST
    Figure CN114116796B_ABST
Patent Text Reader

Abstract

The present invention discloses a distributed cache system for preventing cache stampede, and belongs to the technical field of cloud storage systems. The distributed cache system for preventing cache stampede of the present invention comprises a protocol parser, a cache processor, a key counter, a control configuration item, a current limiter and a protocol generator, wherein the protocol parser, the cache processor and the protocol generator are components of the distributed cache system, and the key counter, the control configuration item and the current limiter are components added to solve the problem of cache stampede. The distributed cache system for preventing cache stampede of the present invention can effectively avoid the problem of cache stampede caused when the cache in the distributed cache system is not pre-stored or suddenly fails, and has a good value for promotion and application.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of cloud storage systems, and in particular provides a distributed cache system for preventing cache stampede. Background Art

[0002] In the computer field, cache is a software / hardware component used to temporarily store data so that future requests can get faster responses. The data stored in the cache can be the results of previous calculations or data in any other storage facility. Distributed cache is an extension of the traditional cache used locally alone. It can be deployed across multiple servers to expand the cache capacity and data transmission capabilities. Distributed cache is mainly used to temporarily store application data or session data of web servers. It is used in business scenarios with high concurrency and large read requests and has been widely used by Internet companies in recent years.

[0003] Key: The cache system discussed in the present invention is a key-value cache. The key is a unique identifier for storing data. The cache client saves or accesses the data in the cache through the key. The key of each data is different and unique.

[0004] Value: The content corresponding to the key in the cache, which is a copy of the data in the cache.

[0005] Hit: refers to the data request being retrieved from the cache and the required data is returned; otherwise it is a miss.

[0006] Timeout: refers to the time when the cached data exists has reached the maximum lifetime allowed by the system for the cached data.

[0007] Elimination: refers to the cache system discarding certain cached data (overwritten by new data) according to a certain strategy.

[0008] Invalidation: refers to the data in the cache being eliminated because the preset timeout period has been reached or because the cache system storage space has reached the upper limit and is eliminated according to the preset elimination policy.

[0009] Penetration: refers to the phenomenon that the data queried by the query request does not hit the cache and instead directly queries the persistence layer.

[0010] Breakdown: refers to the phenomenon that a large number of query requests directly request the database at the moment when the cache fails.

[0011] Avalanche: refers to the phenomenon that a large number of data in the cache become invalid, causing a huge number of cache client processes to penetrate the cache, causing excessive pressure on the database or even downtime.

[0012] Cascading system failures refer to a situation in which the failure of one or some components in a system with multiple related components can recursively cause the failure of other components one after another. Cache stampede is a cascading system failure that occurs in a high-load concurrent computing system with a cache mechanism. Summary of the invention

[0013] The technical task of the present invention is to provide a distributed cache system for preventing cache trampling, which can effectively avoid the cache trampling problem caused by the cache not being pre-stored or suddenly failing in the distributed cache system, in response to the above-mentioned problems.

[0014] To achieve the above object, the present invention provides the following technical solutions:

[0015] A distributed cache system for preventing cache trampling includes a protocol parser, a cache processor, a key counter, a control configuration item, a current limiter and a protocol generator, wherein the protocol parser, the cache processor and the protocol generator are components of the distributed cache system, and the key counter, the control configuration item and the current limiter are components added to solve the cache trampling.

[0016] Preferably, the protocol parser is responsible for receiving and parsing cache client request messages to obtain key values ​​to be saved or data keys to be queried.

[0017] Preferably, the cache processor is used to write or query key values ​​according to user requests.

[0018] Preferably, the key counter is used to count the keys requested by the user but not present in the cache.

[0019] Preferably, the key counter is responsible for recording the number of cache key miss requests when the cache misses, and the number is key value data with an integer value, which is a counting entry and is used by the current limiter to determine the current limiting condition.

[0020] When a key is not hit for the first time, the key counter creates a count entry for the cache key and adds 1 to it. When the key is not hit again, the key counter directly adds 1 to it (if it is Redis, you can call the incr command to conveniently implement it in one step, that is: incrkeyname_key_count1, where keyname is the name of the key that was not hit, and _key_count is the suffix of the key of the key counter count entry).

[0021] When a cache client that receives a miss query finds the value of the data key and writes it to the cache, if the cache client queries again and hits it, the key count entry in the key counter is cleared, thereby reclaiming memory resources and preventing the counter area from occupying memory indefinitely. If no cache client can write the data key value, the key counter clears the key entry according to the maximum request count set in the control configuration item.

[0022] Preferably, the control configuration items are divided into cache system configuration items and client configuration items. The cache system configuration items include the maximum key count value, the peak number of requests and the maximum number of concurrent databases; the client configuration items include the commitment limit and the maximum number of committed retries.

[0023] The maximum key count value (key-counter-max-count) is used to control the count of key counter entries. For example, when key-counter-max-count is 100, the key counter will automatically clear the count entry of a key when the key is not hit for the 101st time.

[0024] Peak request count (peak-request-count) is used by the rate limiter to calculate the commitment value, which can generally be estimated by the number of daily peak active users of the application system.

[0025] The maximum concurrency of the database (max-concurrency) is the maximum single concurrency that the database system can withstand. To ensure the safe and stable operation of the database system, you can set a value slightly smaller than the actual maximum concurrency. It is also used for the calculation of the commitment value by the rate limiter.

[0026] The promise limit (promise-limit) and the maximum number of promise retries (max-promise-retry) are used by the cache client to calculate the time interval after a request misses through the time hash function. The maximum number of promise retries is used to control the number of retries. When max-promise-retry is 0, it means no retry. When it is 3, it means that the cache client will initiate a maximum of 3 retries when receiving the promise response. If the required data is still not obtained after 3 retries, it is considered that the database has failed. The client that released the missed request before the promise cannot successfully refresh the cache. At this time, the attempt should be abandoned and handled as a database connection timeout.

[0027] Preferably, the limiter compares the key count with a count threshold set in a configuration, and classifies the non-existent keys according to the comparison result.

[0028] Preferably, a current limiter is used for current limiting control. When a cache miss occurs, the current limiter compares the count entry of the miss in the key counter with the commitment threshold in the control configuration item. If the count entry value is greater than the commitment threshold, the commitment is calculated and returned, otherwise a miss is directly returned.

[0029] The commitment value can be determined by the following hash function:

[0030] p=H(n) (1)

[0031] Where n is the count entry value, that is, the number of missed requests, and H(n) is a hash function of the count entry value, including but not limited to hash functions commonly used in computer science.

[0032] Preferably, the protocol generator is used to encapsulate the return data into a TCP message according to a specific cache protocol and send it to the cache client.

[0033] This distributed cache system for preventing cache stampede extends a commitment protocol based on the traditional cache system. Taking the RESP protocol as an example, when the cache misses and the rate limiter determines that the request needs to return a commitment, the protocol message is as follows:

[0034] +Promise 7\r\n, this message indicates that the client receiving this promise is the 7th cache client to receive the promise.

[0035] The cache client extends the commitment processing and retry strategy. When the client receives the commitment agreement, it calculates the waiting time for the retry request based on the commitment value and the commitment limit in the configuration item. The formula is as follows:

[0036] T w =L p -p*t n (2)

[0037] Where T w The time the cache client needs to wait for retry, in milliseconds (ms), is called the promised wait time. p is the commitment limit, which is the maximum response time that the client can accept, in milliseconds (ms). p is the commitment value, t n is the average response time of a single data query in the database, p*t n is called the commitment time. When the commitment time is less than the commitment limit, T w A positive number indicates that the promise is acceptable and the cache client will w Retry request query cache after time; when the promised time is greater than the promised limit, T wIf it is a negative number, it means that the commitment is unacceptable, the cache client will give up retrying, and will no longer request the database, and directly throw a timeout exception (which can be handled as a database connection timeout, or directly return a null value or default value for the requested data). The client can be implemented in the form of a lib dependency library of applications such as C / C++ and Java.

[0038] A distributed cache system that prevents cache stampede. In the query scenario when the cache hits (i.e., the key exists and has not expired), the protocol parser parses the cache client request message to obtain the request key, the cache processor queries and obtains the data, and the data is encapsulated in a TCP message by the protocol generator and returned to the cache client in response.

[0039] Query scenario when the cache misses (key does not exist) or is invalid (key expires or is eliminated): the protocol parser parses the cache client request message to obtain the request key, the cache processor queries the miss cache, stores the request key in the counter, and the rate limiter compares the number of requests in the counter with the set miss threshold in the extended configuration: if the number of requests is less than the threshold, a miss is directly returned to the protocol generator; if the number of requests is greater than the threshold, a commitment protocol is returned to the protocol generator, and the protocol generator encapsulates the TCP message and returns the response to the cache client.

[0040] When multiple cache clients request data from this system at the same time, if the data key value does not exist, only a few (less than the threshold number set by the extended configuration) cache clients will receive a miss return message, and the other cache clients will receive a commitment agreement. The cache client that receives the commitment agreement calculates a retry time based on the committed value through a time hash function, and controls the time and number of request retries to query the cache in combination with the request retry strategy set in its own configuration, thereby achieving active current limiting in massive concurrency scenarios and effectively avoiding cache stampede.

[0041] Compared with the prior art, the distributed cache system for preventing cache stampede of the present invention has the following outstanding beneficial effects: the distributed cache system for preventing cache stampede extends the commitment protocol between the cache system and the cache client, and uses the time hash function to time hash and retry a huge number of requests in a short period of time, which can effectively avoid the cache stampede problem caused by the cache not being pre-stored or suddenly failing in the distributed cache system, and has good promotion and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a working principle diagram of a key counter in a distributed cache system for preventing cache stampede according to the present invention;

[0043] Figure 2 is a schematic diagram of the current limiter activity of the distributed cache system for preventing cache stampede according to the present invention;

[0044] Figure 3 Schematic diagram of client activities of the distributed cache system for preventing cache stampede according to the present invention. DETAILED DESCRIPTION

[0045] The distributed cache system for preventing cache stampede of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

[0046] Example

[0047] like Figure 1 As shown, the distributed cache system for preventing cache trampling of the present invention includes a protocol parser, a cache processor, a key counter, a control configuration item, a current limiter and a protocol generator, wherein the protocol parser, the cache processor and the protocol generator are components possessed by the distributed cache system, and the key counter, the control configuration item and the current limiter are components added to solve the cache trampling.

[0048] The protocol parser is responsible for receiving and parsing cache client request messages to obtain the key value to be saved or the data key to be queried.

[0049] Writing scenario: The protocol parser parses the cache client request message to obtain the request key and value. The cache processor stores the key value in the memory and sends a success signal to the protocol generator. The protocol generator encapsulates the success signal into a TCP message and returns a response to the cache client.

[0050] like Figure 1 As shown, the cache processor is used to write or query key values ​​according to user requests. The key counter is used to count the keys requested by the user but not in the cache. The key counter is responsible for recording the number of cache key miss requests when the cache misses. The number is the key value data with an integer value, which is a counting entry and is used by the current limiter to determine the current limit condition.

[0051] When a key is not hit for the first time, the key counter creates a count entry for the cache key and adds 1 to it. When the key is not hit again, the key counter directly adds 1 to it (if it is Redis, you can call the incr command to conveniently implement it in one step, that is: incrkeyname_key_count1, where keyname is the name of the key that was not hit, and _key_count is the suffix of the key of the key counter count entry).

[0052] When a cache client that receives a miss query finds the value of the data key and writes it to the cache, if the cache client queries again and hits it, the key count entry in the key counter is cleared, thereby reclaiming memory resources and preventing the counter area from occupying memory indefinitely. If no cache client can write the data key value, the key counter clears the key entry according to the maximum request count set in the control configuration item.

[0053] Control configuration items are divided into cache system configuration items and client configuration items. The cache system configuration items include the maximum key count value, peak request number, and maximum database concurrency; the client configuration items include the commitment limit and the maximum committed retry number.

[0054] The maximum key count value (key-counter-max-count) is used to control the count of key counter entries. For example, when key-counter-max-count is 100, the key counter will automatically clear the count entry of a key when the key is not hit for the 101st time.

[0055] Peak request count (peak-request-count) is used by the rate limiter to calculate the commitment value, which can generally be estimated by the number of daily peak active users of the application system.

[0056] The maximum concurrency of the database (max-concurrency) is the maximum single concurrency that the database system can withstand. To ensure the safe and stable operation of the database system, you can set a value slightly smaller than the actual maximum concurrency. It is also used for the calculation of the commitment value by the rate limiter.

[0057] The promise limit (promise-limit) and the maximum number of promise retries (max-promise-retry) are used by the cache client to calculate the time interval after a request misses through the time hash function. The maximum number of promise retries is used to control the number of retries. When max-promise-retry is 0, it means no retry. When it is 3, it means that the cache client will initiate a maximum of 3 retries when receiving the promise response. If the required data is still not obtained after 3 retries, it is considered that the database has failed. The client that released the missed request before the promise cannot successfully refresh the cache. At this time, the attempt should be abandoned and handled as a database connection timeout.

[0058] like Figure 2 As shown, the current limiter compares the key count with the count threshold set in the configuration, and classifies the non-existent keys according to the comparison result. The current limiter is used for current limiting control. When the cache misses, the current limiter compares the count entry of the missed key counter with the commitment threshold in the control configuration item. If the count entry value is greater than the commitment threshold, the commitment is calculated and returned, otherwise a miss is directly returned.

[0059] The commitment value can be determined by the following hash function:

[0060] p=H(n) (1)

[0061] Where n is the count entry value, that is, the number of missed requests, and H(n) is a hash function of the count entry value, including but not limited to hash functions commonly used in computer science.

[0062] Take Facebook as an example. Facebook currently has more than 2.45 billion active users per month worldwide. However, global applications are generally deployed in multiple regions, with each region bearing a portion of the load. For example, the United States, which has the largest number of users, currently has a population of more than 320 million, of which 70% are Facebook users, or about 200 million. Assuming that Facebook has 20 regions in the United States, each region has about 20 million active users per month. Based on an estimate that 10% of users are online at the same time, the instantaneous concurrency of a single region is 2 million. A single-node MySQL database can generally carry a maximum of 1,000 concurrent requests. Using read-write separation or clustering technology, the carrying capacity can be increased to about 5 times, that is, 5,000 concurrent requests. The instantaneous request is 400 times the database carrying capacity, which means that 400 rounds of concurrent requests are required to complete all queries. The problem is then transformed into using a hash function to reduce the frequency of 2 million concurrent database query requests to 50ms per request, requiring a total of 400 rounds of requests.

[0063] Assuming that the average query time for a single piece of data by index query is 50 milliseconds (ms), 400 rounds require 20,000 milliseconds (ms), that is, 2 million requests are divided into 400 rounds of retries, which requires a total of 50ms*400=20,000ms. Assuming that the user's acceptable maximum response time is 5 seconds (s), that is, 5,000 ms, it is equivalent to only satisfying 100 rounds of retries, and the remaining 300 rounds of retries need to be processed by fuses.

[0064] Ideally, the internal network transmission delay of the cluster can be ignored. Without considering the impact of refreshing the count entry value when the key counter reaches the limit value, the hash function H(p) can be simply determined using the following formula:

[0065]

[0066] Where n is the count entry value, that is, the number of requests that have not hit the cache. s This is the single maximum concurrent load capacity of the database cluster when assuming cache penetration, which is 5000 in this example. To perform floor operation, Indicates a database concurrent batch corresponding to a missed request. For example, when n is 1 to 4999, =0. From formula (3), we can see that the commitment value of the 1st to 4999th missed requests is 0, that is, the limiter will return a miss to the 1st to 4999th requests, allowing them to directly access the database to query data and update the cache; and the requests with n of 5000 to 9999 will retry the request cache in the same batch.

[0067] Now, considering that the key counter in the cache will be reset when it reaches the maximum key count value, that is, re-counting, the maximum key count value k and the total number of requests N (500,000 in this example) are introduced into formula (3). When k < N, Indicates how many groups the key counter refresh will divide the requests into, that is, how many requests have the same count entry value. For example, when k is 5000, is 100, which means that every 100 requests in 500,000 requests have the same count entry value. Indicates how many groups of requests with the same count entry value can be processed by the database concurrently, then formula (3) can be evolved into formula (4):

[0068]

[0069] According to the properties of fractions, it can be organized into formula (5):

[0070]

[0071] When k ≥ N, H(n) degenerates into formula (3), that is, H(n) is a piecewise function of the maximum key count value k:

[0072]

[0073] The daily peak concurrent number N in formula (6) and the maximum concurrent number N supported by the database system at a time s That is, the peak request count (peak-request-count) and the maximum database concurrency (max-concurrency) described in the control configuration items in Section 4.3.

[0074] At the same time, it is noted that a reasonable setting of the maximum key count value k of the key counter can trigger multiple batches of miss responses, thereby ensuring that the client can try to read the database multiple times and refresh the data to be queried into the cache.

[0075] The protocol generator is used to encapsulate the returned data into a TCP message according to a specific cache protocol and send it to the cache client. This distributed cache system for preventing cache stampede extends a commitment protocol based on the traditional cache system. Taking the RESP protocol as an example, when the cache misses and the current limiter determines that this request needs to return a commitment, the protocol message is as follows:

[0076] +Promise 7\r\n, this message indicates that the client receiving this promise is the 7th cache client to receive the promise.

[0077] The cache client extends the commitment processing and retry strategy. When the client receives the commitment agreement, it calculates the waiting time for the retry request based on the commitment value and the commitment limit in the configuration item. The formula is as follows:

[0078] T w =L p -p*t n (2)

[0079] Where T w The time the cache client needs to wait for retry, in milliseconds (ms), is called the promised wait time. p is the commitment limit, which is the maximum response time that the client can accept, in milliseconds (ms). p is the commitment value, t n is the average response time of a single data query in the database, p*t n is called the commitment time. When the commitment time is less than the commitment limit, T w A positive number indicates that the promise is acceptable and the cache client will w Retry request query cache after time; when the promised time is greater than the promised limit, T w If it is a negative number, it means that the promise is unacceptable, the cache client will give up retrying, and will no longer request the database, and directly throw a timeout exception (it can be handled as a database connection timeout, or a null value or default value can be directly returned for the requested data). Figure 3 The client can be implemented in the form of lib dependency class libraries of applications such as C / C++ and Java.

[0080] A distributed cache system that prevents cache stampede. In the query scenario when the cache hits (i.e., the key exists and has not expired), the protocol parser parses the cache client request message to obtain the request key, the cache processor queries and obtains the data, and the data is encapsulated in a TCP message by the protocol generator and returned to the cache client in response.

[0081] Query scenario when the cache misses (key does not exist) or is invalid (key expires or is eliminated): the protocol parser parses the cache client request message to obtain the request key, the cache processor queries the miss cache, stores the request key in the counter, and the rate limiter compares the number of requests in the counter with the set miss threshold in the extended configuration: if the number of requests is less than the threshold, a miss is directly returned to the protocol generator; if the number of requests is greater than the threshold, a commitment protocol is returned to the protocol generator, and the protocol generator encapsulates the TCP message and returns the response to the cache client.

[0082] When multiple cache clients request data from this system at the same time, if the data key value does not exist, only a few (less than the threshold number set by the extended configuration) cache clients will receive a miss return message, and the other cache clients will receive a commitment agreement. The cache client that receives the commitment agreement calculates a retry time based on the committed value through a time hash function, and controls the time and number of request retries to query the cache in combination with the request retry strategy set in its own configuration, thereby achieving active current limiting in massive concurrency scenarios and effectively avoiding cache stampede.

[0083] The embodiments described above are only preferred specific implementations of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A distributed cache system for preventing cache stampede, characterized in that: The invention comprises a protocol parser, a cache processor, a key counter, a control configuration item, a current limiter and a protocol generator, wherein the protocol parser, the cache processor and the protocol generator are components of a distributed cache system, the key counter, the control configuration item and the current limiter are components added to solve cache stampede, wherein the distributed cache system for preventing cache stampede extends a commitment protocol between the cache system and the cache client, and uses a time hash function to time hash and retry a huge number of requests in a short period of time, and the protocol parser is responsible for receiving and parsing cache client request messages to obtain key values ​​to be saved or data keys to be queried; The cache processor is used to write or query key values ​​according to user requests; the key counter is used to count keys requested by users but not present in the cache; the key counter is responsible for recording the number of cache key miss requests when the cache misses, which is key value data with an integer value, a counting entry, and is used by the current limiter to determine the current limiting condition; the control configuration items are divided into cache system configuration items and client configuration items, and the cache system configuration items include the maximum key count value, the peak request number and the maximum database concurrency; the client configuration items include the commitment limit and the maximum committed retry number; the current limiter compares the key count with the count threshold set by the configuration, and classifies the non-existent keys according to the comparison result; the current limiter is used for current limiting control, and when the cache misses, the current limiter compares the missed count entry in the key counter with the commitment threshold in the control configuration item, and calculates and returns the commitment if the count entry value is greater than the commitment threshold, otherwise directly returns a miss; the protocol generator is used to encapsulate the return data into a TCP message according to a specific cache protocol and send it to the cache client.