Dynamic hot data caching method

Abstract

The present invention discloses a dynamic hot data caching method, which adjusts each cached timestamp by comparing the weight of the access parameter with the weight of the pre-set access parameter, that is, the weight of the cache hit rate, the weight of the access frequency with the preset weight, and sets the cache parameter in combination with the historical cache parameter. The present invention can automatically identify hot data and automatically cache it, and can greatly reduce the pressure on the server through hot data prejudgment; the cache expiration time is no longer fixed, but is dynamically optimized, which can avoid the appearance of dirty data, continuously adjust the cache life cycle, and find the optimal cache life cycle.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a dynamic hot data caching method. Background Art

[0002] In web scenarios, in order to cope with high concurrency and reduce server pressure, it is necessary to cache hot data, that is, to change the storage form of data that needs to be read frequently from files, databases, etc. to memory storage. Therefore, there are memory databases such as membercache and redis to reduce I / O consumption.

[0003] The traditional technical solution is: developers determine which data is hot, and cache it by writing it into the business code. At the same time, they need to monitor database data updates to update the cache and prevent dirty data; use servers such as nginx for dynamic caching; and use the elimination strategy of the redis database to optimize the storage of hot data.

[0004] Traditional technical solutions have the following disadvantages: they rely on developers to manually identify hot data and integrate cache code into business code; the cache time is fixed and will not be updated before the cache time expires, and it is only applicable to static data; as a database, the redis database can only optimize stored data, and the expiration time setting is mechanical. Summary of the invention

[0005] The purpose of the present invention is to provide a dynamic hot data caching method to solve the above problem, comprising the following steps:

[0006] S1: read the access request and obtain the access parameters;

[0007] S2: Determine whether the data of the access request exists in the cache. If so, record the access parameters and go to S3. Otherwise, if there is no cache, go to S4.

[0008] S3: Check whether the cache is valid. If the cache is valid, go to S7. Otherwise, if the cache is invalid, delete the cache and go to S4.

[0009] S4: pre-set cache hit rate weight and access frequency weight;

[0010] S5: record access parameters and update cache log data;

[0011] S6: Calculate the cache hit rate weight and the access frequency weight, and compare the current cache hit rate weight and the access frequency weight with the preset cache hit rate weight and the access frequency weight. If the current cache hit rate weight and the access frequency weight are greater than the preset cache hit rate weight and the access frequency weight, then establish a cache, set cache parameters, and go to S7; otherwise go to S7;

[0012] S7: Return a response.

[0013] The beneficial effects of the present invention are as follows: the present invention can automatically identify hot data and automatically cache it, and can greatly reduce server pressure through hot data prediction; the cache expiration time is no longer fixed, but is dynamically optimized, which can avoid the occurrence of dirty data, and can continuously adjust cache parameters to find the optimal cache life cycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0015] The present invention will be further described below in conjunction with the accompanying drawings:

[0016] As attached Figure 1 As shown, the dynamic hot data caching method of the present invention comprises the following steps:

[0017] S1: read the access request and obtain the access parameters;

[0018] S2: Determine whether the data of the access request exists in the cache. If so, record the access parameters and go to S3. Otherwise, if there is no cache, go to S4.

[0019] S3: Check whether the cache is valid. If the cache is valid, go to S7. Otherwise, if the cache is invalid, delete the cache and go to S4.

[0020] S4: pre-set cache hit rate weight and access frequency weight;

[0021] S5: record access parameters and update cache log data;

[0022] S6: Calculate the cache hit rate weight and the access frequency weight, and compare the current cache hit rate weight and the access frequency weight with the preset cache hit rate weight and the access frequency weight. If the current cache hit rate weight and the access frequency weight are greater than the preset cache hit rate weight and the access frequency weight, then establish a cache, set cache parameters, and go to S7; otherwise go to S7;

[0023] S7: Return a response.

[0024] Specifically, the access parameters include a start timestamp, a number of accesses, a hash value of a response, a number of cache hits, a cache hit rate, an access frequency, a last cache miss timestamp, and a cache life cycle.

[0025] Specifically, the detection of whether the cache is valid in S3 is to compare the hash value in the cache log data with the hash value of the response. If they are the same, the cache is valid, otherwise the cache is invalid.

[0026] Specifically, the weights of the access parameters in S2 at least include the weight of the cache hit rate and the weight of the access frequency.

[0027] Specifically, the detection of cache validity in S3 is a random detection.

[0028] Specifically, the cache parameters include a start timestamp, a cache expiration time, a cache life cycle, and a last cache miss timestamp.

[0029] Specifically, the specific process of setting the cache parameter is to set the cache parameter to take the median of the historical data.

[0030] Starting from the interface access, parse and record the access parameters, including the start timestamp, number of accesses, response hash value, cache hit count, last cache miss timestamp and cache lifecycle.

[0031] Determine whether there is a cache. If there is a cache, record the access parameters, randomly check the validity of the cache, calculate and extract the hash value in the cache log data and the hash value of the response, and compare them. If they are the same, the cache is valid, otherwise the cache is invalid. If the cache is valid, use the cache and return the response. If the cache is invalid, delete the cache. If there is no cache or the cache is invalid, extract the number of cache hits, the number of accesses, and the time window, calculate the cache hit rate based on the ratio of the number of cache hits to the number of accesses, and calculate the access frequency based on the ratio of the number of accesses to the time window. The access frequency is used to identify hot data. Preset the weights of cache hit rate and access frequency, and use the cache hit rate weight as the priority weight; record access parameters and update them to cache log data; cache setting parameters include start timestamp, cache expiration time, cache life cycle and last cache miss timestamp; calculate the current cache hit rate weight and access frequency weight through access parameters, and compare them with the preset cache hit rate weight and access frequency weight. If they exceed the preset cache hit rate weight and access frequency weight, establish a cache, set cache parameters, namely adjust timestamp, cache expiration time, cache life cycle and last cache miss timestamp, and take the median of historical data; update cache log data and return a response.

[0032] In combination with historical access parameters, the weights of pre-set access parameters are adjusted. By comparing the weights of access parameters with the weights of pre-set access parameters, that is, the weights of cache hit rate and access frequency with the preset weights, the timestamps of the cache are adjusted. The cache parameters are set in combination with historical cache parameters to achieve the purpose of improving cache hit rate and optimizing cache, and realizing closed-loop feedback of the whole process.

[0033] The present invention can adjust the access frequency weight through the number of accesses and time window parameters in the log, analyze the existing hot data parameters, extract the characteristics of the hot data parameters, perform data preprocessing on the high concurrency time period and number of requests and the request interface, and adjust the cache parameters before the high concurrency comes; the cache life cycle uses the number of historical data, and as the life cycle parameters continue to increase, the cache expiration time is continuously adjusted, which is conducive to finding the optimal cache life cycle.

[0034] The present invention can automatically identify hot data and automatically cache it, and can greatly reduce server pressure through hot data prediction; the cache expiration time is no longer fixed, but is dynamically optimized, which can avoid the occurrence of dirty data, and can continuously adjust cache parameters to find the optimal cache life cycle.

[0035] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. All technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A dynamic hot data caching method, characterized in that: The steps include: S1: Read the access request and obtain access parameters; the access parameters include the start timestamp, the number of accesses, the hash value of the response, the number of cache hits, the cache hit rate, the access frequency, the last cache miss timestamp and the cache life cycle; S2: Determine whether the data of the access request exists in the cache. If so, record the access parameter and go to S3. Otherwise, go to S4. The weight of the access parameter in S2 at least includes the weight of the cache hit rate and the weight of the access frequency. S3: Check whether the cache is valid. The check whether the cache is valid is to compare the hash value in the cache log data with the hash value of the response. If they are the same, the cache is valid, otherwise the cache is invalid. If the cache is valid, go to S7, otherwise if the cache is invalid, delete the cache and go to S4. S4: pre-set cache hit rate weight and access frequency weight; S5: record access parameters and update cache log data; S6: Calculate the cache hit rate weight and access frequency weight, compare the current cache hit rate weight and access frequency weight with the preset cache hit rate weight and access frequency weight, if the current cache hit rate weight and access frequency weight are greater than the preset cache hit rate weight and access frequency weight, then establish a cache, set cache parameters, and go to S7; otherwise go to S7; the cache parameters include a start timestamp, a cache expiration time, a cache life cycle, and a last cache miss timestamp; the specific process of setting the cache parameters is to set the cache parameters to take the median of historical data; S7: Return a response.

2. The dynamic hot data caching method according to claim 1, characterized in that: The detection of cache validity in S3 is a random detection.

Images