Cache data management method and device, computer equipment and storage medium

Abstract

The invention relates to a cache data management method and device, computer equipment and a storage medium. The method comprises the following steps: counting the access request frequency of each piece of data in unit time, and evaluating the access heat of the data according to the access request frequency; in response to the received access request of the target data and the frequency of the access request of the target data reaching a preset cache threshold, establishing a cache of the target data; determining a cache diffusion strategy according to the access popularity of the target data, determining a target diffusion node according to the cache diffusion strategy, and diffusing the target data to a cache service of the target diffusion node; and in response to the condition that the target data meets the preset failure condition, diffusion stopping and / or cache deleting of the target data are / is executed on the target data. By adopting the method, whether the cache needs to be established or not and whether cache diffusion needs to be performed or not can be determined according to the data access popularity, and the situation that low-frequency data occupy the cache space is avoided.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a cache data management method, device, computer equipment and storage medium. Background Art

[0002] In modern web systems, in order to improve data access speed and reduce the pressure on the backend database, a cache layer is usually designed between the web layer and the database layer. This design allows the web layer to obtain data from the cache layer first when querying data, and only access the database when the cache misses, update the data to the cache and return the result.

[0003] There are two common architectures for Web systems that support high availability. Figure 1 As shown in the figure, the web layer and cache layer of each node are independent of each other, the database forms a master-slave cluster, and each node can provide services to the outside world. In this architecture, if the data does not exist in the cache, the system will access the database and put the data into the cache, but the data will be put into the cache as long as it is accessed once, which may cause a large amount of infrequently accessed data in the cache, occupying a large space and causing resource waste. In addition, since the cache layers of each node are independent of each other, users may repeatedly query data and update the cache when accessing different nodes, affecting the interface access speed.

[0004] The second architecture is Figure 2 As shown in the figure, the cache layer of each node forms a distributed cache cluster. In this design, after a user visits any node, the data will be saved in the distributed cache. When the user visits other nodes again, the data can be directly obtained from the distributed cache without having to establish the cache again, thereby improving the access speed. However, the cache layer of each node may occupy the same memory space. Even if some nodes do not receive requests, their cache layers may store a large amount of data, resulting in a waste of memory resources. Summary of the invention

[0005] Based on this, it is necessary to provide a cache data management method, device, computer equipment and storage medium that can realize adaptive diffusion of cache data in response to the above technical problems.

[0006] In one aspect, a cache data management method is provided, the method comprising:

[0007] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency;

[0008] In response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data;

[0009] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0010] In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data.

[0011] In one embodiment, determining a cache diffusion strategy according to the access popularity of the target data, determining a target diffusion node according to the cache diffusion strategy, and diffusing the target data to a cache service of the target diffusion node includes:

[0012] In response to the access request frequency of the target data being less than a first frequency threshold, storing the cache of the target data in the current node cache service without spreading, wherein the first frequency threshold is greater than the preset cache threshold;

[0013] In response to the access request frequency of the target data being greater than or equal to a first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0014] In one embodiment, the method further comprises:

[0015] Comparing the access request frequency of the target data with a preset cache threshold;

[0016] In response to the frequency of access requests to the target data being less than a preset cache threshold, the cache of the target data is not saved, and the database is directly accessed.

[0017] In one of the embodiments, in response to the frequency of access requests to the target data being greater than or equal to a first frequency threshold, matching the target data with candidate nodes to determine at least one target diffusion node includes:

[0018] Collect resource usage parameters of each node and calculate the resource utilization rate of each node;

[0019] Filter nodes whose resource utilization is less than a preset load threshold and add them to the candidate node list;

[0020] In response to the access request frequency of the target data being greater than or equal to a first frequency threshold and less than a second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list;

[0021] Or, record the cache hit situation of each node and calculate the cache hit rate of the node;

[0022] Calculating cache gains of all candidate nodes in the candidate node list according to the access popularity of the target data, the resource utilization rate of the node, and the cache hit rate;

[0023] Acquire a candidate node whose cache gain is greater than a preset diffusion threshold as a first target diffusion node;

[0024] In response to the access request frequency of the target data being greater than or equal to a second frequency threshold, randomly selecting a plurality of second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes;

[0025] Or, a candidate node whose cache gain is greater than a preset diffusion threshold is obtained as a second target diffusion node.

[0026] In one of the embodiments, in response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data includes:

[0027] Periodically scan all cached data to determine whether the cached data has exceeded its expiration time;

[0028] In response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data;

[0029] In response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache;

[0030] In response to the current access heat being greater than or equal to a preset cache threshold, the cache of the target data is not deleted, and it is determined whether to stop diffusion of the target data according to the current access heat.

[0031] In one embodiment, in response to the current access heat being greater than or equal to a preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data according to the current access heat, includes:

[0032] In response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from a cache of the first target diffusion node;

[0033] In response to the current access request frequency being less than a second frequency threshold and greater than or equal to a first frequency threshold, the target data is deleted from the cache of the second target diffusion node, and the first target diffusion node is confirmed based on the current access request frequency, and the target data is diffused to the cache service of the first target node.

[0034] In one embodiment, the method further comprises:

[0035] In response to the cache data amount reaching an upper threshold, obtaining data access heat, data size, and remaining validity period of the cache data;

[0036] Setting the elimination priority of the cached data according to the data access popularity, data size and remaining validity period of the data;

[0037] The target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0038] In another aspect, a cache data management device is provided, the device comprising:

[0039] An evaluation module, used to count the access request frequency of each data within a unit time, and evaluate the access popularity of the data according to the access request frequency;

[0040] A cache module, configured to establish a cache of the target data in response to receiving an access request for the target data and the access request frequency of the target data reaching a preset cache threshold;

[0041] A diffusion module, used to determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0042] The deletion module is used for stopping diffusion of the target data and / or deleting the cache of the target data in response to the target data satisfying a preset expiration condition.

[0043] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0044] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency;

[0045] In response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data;

[0046] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0047] In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data.

[0048] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0049] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency;

[0050] In response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data;

[0051] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0052] In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data.

[0053] The cache data management method, device, computer equipment and storage medium above count the access request frequency of each data in a unit time, evaluate the access popularity of the data according to the access request frequency, and decide whether to establish a cache and whether to perform cache diffusion according to the access popularity, so as to ensure that only high-frequency access data is cached, avoid low-frequency data occupying cache space, speed up data access speed of other nodes, and reduce the pressure on the database. In addition, the operation of stopping diffusion or deleting cache according to the preset failure condition can ensure that cache resources are released in time to avoid resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of a local cache architecture in the prior art;

[0055] Figure 2 A schematic diagram of a distributed cache architecture in the prior art;

[0056] Figure 3 A schematic diagram of a flow chart of a cache data management method in one embodiment;

[0057] Figure 4 The following is a flow chart of the steps for obtaining cold data in one embodiment;

[0058] Figure 5 A schematic diagram of a process flow for caching warm data in one embodiment;

[0059] Figure 6 A schematic diagram of a process flow of hot data cache diffusion steps in one embodiment;

[0060] Figure 7 It is a schematic diagram of the process flow of caching and spreading steps of extremely hot data in one embodiment;

[0061] Figure 8 is a structural block diagram of a cache data management device in one embodiment;

[0062] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] In one embodiment, Figure 3-7 As shown, a cache data management method is provided, comprising the following steps:

[0065] Step S1, counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency.

[0066] Specifically, the access statistics aspect is implemented in the web layer. This aspect intercepts the method calls of the web layer and uses a counter or sliding window algorithm to record the number of calls of each intercepted method per unit time, thereby obtaining the access request frequency of each data. And because it is implemented by the aspect, there is no need to modify the existing code logic.

[0067] For example, the access heat of data is evaluated based on the statistical access request frequency n. If n is less than 1 time / second, the data heat is considered to be cold; if n is 1-100 times / second, the data heat is considered to be warm; if n is 100-300 times / second, the data heat is considered to be hot; if n is greater than 300 times / second, the data heat is considered to be extremely hot. Subsequently, the corresponding cache diffusion strategy can be selected according to the access heat of the data, and support can be provided for cache failure.

[0068] Step S2: in response to receiving an access request for target data and the access request frequency of the target data reaching a preset cache threshold, establishing a cache for the target data.

[0069] In one possible implementation, the access request frequency of the target data is compared with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, the cache of the target data is not saved and the database is directly accessed.

[0070] Specifically, when an access request for target data is received, the access popularity of the data is first checked. If the access popularity is lower than the preset cache threshold, the database is directly accessed to obtain the data; if the access popularity exceeds the cache threshold, the regular cache process is followed to check whether the target data exists in the cache. If so, the data is returned directly from the cache; if not, the target data is obtained from the database and saved in the cache of the current node.

[0071] For example, Figure 4 As shown in the figure, the cache threshold is set to 1 time / second. When the access request frequency of the target data is less than 1 time / second, that is, the access heat of the target data is cold, the database can fully support queries of this frequency and will not save the data (method return value) to the cache. Therefore, no cache is established and the database is accessed directly.

[0072] like Figure 5 As shown in the figure, when the access request frequency of the target data is in the range of 1-100 times / second, that is, the access heat of the target data is warm, the database will face greater access pressure and a cache needs to be established. Therefore, the target data is saved in the current node cache service in the form of key / value, where the key is the called method name + method input parameters, and the value is the method return value. Subsequent access is directly obtained from the cache, thereby reducing database pressure.

[0073] Based on the above steps, whether to establish a cache is determined according to the data access popularity. If the data access frequency is low, no cache will be established. Since the access popularity of more data in the system is low-frequency data, it will not be saved in the cache. Therefore, the cache space occupied by low-frequency data can be reduced, thereby effectively utilizing limited cache resources and ensuring that data with high access frequency is responded to quickly.

[0074] Step S3: determining a cache diffusion strategy according to the access popularity of the target data, determining a target diffusion node according to the cache diffusion strategy, and diffusing the target data to the cache service of the target diffusion node.

[0075] In one possible implementation, the cache diffusion strategy is determined according to the access popularity of the target data, and the target diffusion node is determined according to the cache diffusion strategy. The specific steps of diffusing the target data to the cache service of the target diffusion node include: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is saved in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0076] Among them, the preset cache threshold (for example, 1 time / second) is used to distinguish between cold data and hot data, so as to determine whether it is necessary to establish a cache, and the first frequency threshold (for example, 100 times / second) is used to determine whether it is necessary to spread the data to other nodes. When the access request frequency of the target data is greater than or equal to the first frequency threshold, for example, 100 times / second, the access heat of the data is hot or extremely hot. Since the data request is very frequent, other nodes are likely to use the data. Therefore, in addition to establishing a cache, the data needs to be spread to other node caches so that when requesting to access other nodes, the data can be directly obtained from the cache, thereby improving the response speed and reducing the database pressure.

[0077] Based on the above steps, frequently requested data is automatically diffused and cached in advance. When other nodes receive a request for the data, they can directly obtain the data from the local cache without sending a query request to the database, ensuring that data with high access frequency is responded to quickly and speeding up data access for other nodes.

[0078] In one possible implementation manner, in response to the target data access request frequency being greater than or equal to the first frequency threshold, matching the target data with candidate nodes, and determining at least one target diffusion node include: collecting resource usage parameters of each node and calculating the resource utilization of each node; screening nodes whose resource utilization is less than a preset load threshold to add to the candidate node list; in response to the target data access request frequency being greater than or equal to the first frequency threshold and less than the second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording the cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list according to the access heat of the target data, the resource utilization rate of the node and the cache hit rate; obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the target data access request frequency being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0079] Specifically, it is possible to obtain indicator information of the target data and determine the data capacity of the target data to be diffused according to the indicator information, the indicator information including the number of data requests, data size, and data bandwidth; obtain load information of each node, the load information including the equipment construction capacity and the current carrying capacity of the cache device in the node; predict the expected carrying capacity of each node after adding the cache data of the target data according to the data capacity to be diffused and the current carrying capacity; determine the capacity utilization of each node after adding the target data according to the expected carrying capacity and the equipment construction capacity; traverse all nodes, determine the nodes whose capacity utilization is less than the load threshold as candidate nodes, and establish a list of candidate nodes.

[0080] For example, Figure 6 As shown, the cluster nodes include node 1, node 2, and node 3. Assuming that these nodes all meet the resource utilization conditions, the second frequency threshold is set to 300 times / second. When node 1 receives a request, if the target data is in the range of 1-300 times / second, that is, the access heat is hot, a node can be randomly selected from node 2 and node 3, such as node 2, and then the data can be synchronized to the node 2 cache service; the cache gains of nodes 2 and 3 can also be calculated respectively, and the node with the maximum cache gain can be selected. At this time, the node 2 cache service can also provide services to the outside world. If a request is made to access node 2, the data can be directly obtained from the cache, which improves the response speed and reduces the database pressure.

[0081] like Figure 7 As shown in the figure, if the access request frequency of the target data is greater than 300 times / second, that is, the access popularity is extremely got, cache diffusion is also performed, but more nodes will be selected to synchronize the data to the cache services of node 2 and node 3 to further disperse the load.

[0082] It should be noted that the number of target diffusion nodes can be determined based on the total number of cluster nodes. For example, if the cluster has 10 nodes, for data items with hot access (100-300 times / second), 10% to 20% of the nodes can be selected as the first target diffusion nodes; for data items with extremely hot access (greater than or equal to 300 times / second), 30% to 50% of the nodes can be selected as the second target diffusion nodes. It is also possible to set a diffusion threshold so that all nodes whose calculated cache gain is greater than the diffusion threshold are used as target diffusion nodes. At this time, for data items with extremely hot access, the calculated cache gain is naturally greater than that of data items with hot access. Therefore, when selecting nodes by calculating the cache gain, the number of second target diffusion nodes is still greater than the number of first target diffusion nodes.

[0083] Based on the above steps, by screening nodes with low resource utilization to establish a candidate node list, we can make full use of idle resources, and determine the number of target diffusion nodes by accessing popularity or calculating cache gain, so as to diffuse high-frequency data to the cache services of multiple nodes, ensuring that high-frequency data is responded to quickly, while reducing the pressure on the database.

[0084] Step S4: in response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting the cache of the target data.

[0085] In one possible implementation, in response to the target data satisfying a preset expiration condition, the specific steps of stopping diffusion of the target data and / or deleting the cache of the target data include: periodically scanning all cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0086] Specifically, in order to prevent data from occupying cache space all the time, a cache expiration mechanism is designed, that is, each data is set with an expiration time when it is added to the cache, and a cache scanning thread is introduced. This thread periodically (for example, every 5 seconds) scans all cached data. If the scanning thread finds that the data has expired and the data heat is reduced, the cache invalidation mechanism will be triggered to delete the cache of low-frequency data to reduce space usage. If it is found that a cached data has expired, but the heat of the cached data is warm or above, the data will not be deleted from the cache this time, and the expiration time of the cached data will be automatically extended, which is equivalent to exempting the expired data from this scan.

[0087] Based on the above steps, by periodically scanning the entire cached data and checking whether the data has exceeded the expiration time, the cache space occupied by data that is no longer frequently used can be released in a timely manner. For data that is still highly accessed, its cached data is temporarily retained even if it exceeds the preset expiration time, thereby improving the cache hit rate and reducing the pressure on the database.

[0088] In one possible implementation, in response to the current access heat being greater than or equal to a preset cache threshold, the cache of the target data is not deleted, and the specific steps of determining whether to stop spreading the target data according to the current access heat include: in response to the current access request frequency being less than a first frequency threshold, stopping the spread of the target data, and deleting the target data from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold, and greater than or equal to the first frequency threshold, deleting the target data from the cache of the second target diffusion node, confirming the first target diffusion node according to the current access request frequency, and diffusing the target data to the cache service of the first target node.

[0089] For example, if the data access heat decreases and becomes warm, node 1 will no longer diffuse cache information to the outside. Since node 2 no longer receives cache data diffused by node 1, the data will automatically become invalid when the expiration time comes and will be deleted from the cache of node 2. If the data heat decreases and becomes hot, node 1 will reduce the diffusion of cache data to nodes 2 and 3 to only one node (such as node 2). Since node 3 no longer receives cache data diffused by node 1, the data will automatically become invalid when the expiration time comes and will be deleted from the cache of node 3.

[0090] Based on the above steps, the introduction of cache invalidation mechanism can decide whether to stop cache diffusion or delete cached data according to the changes in data access popularity, and adjust the diffusion range of cached data, which can better adapt to changes in data access patterns and improve cache hit rate and system response speed.

[0091] In one possible implementation, in response to the amount of cached data reaching an upper threshold, the data access heat, data size and remaining data validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining data validity period; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0092] Specifically, cache data is eliminated step by step according to data access heat, data size, and data remaining validity period. For example, if warm, hot, and extremely hot data exist in the cache at the same time, and the cache capacity is insufficient, warm-level data will be eliminated first. If the space is still insufficient, hot-level data will be eliminated secondly, and extremely hot data will be eliminated last, until the cache data volume is less than the upper threshold. When eliminating data of the same access heat level, data with large space occupation will be preferred. For example, when eliminating warm-level data, there are 5 warm-level data, a, b, c, d, and e, and the capacity occupation is 20k, 2k, 3k, 4k, and 5k respectively. When eliminating data, data a will be eliminated first, because eliminating a alone will release 20k space, while eliminating b, c, d, and e will only release 14k space in total, and the impact of eliminating one data is less than eliminating four data, so eliminating data a has the highest cost-effectiveness. If the size of more data is the same, the remaining validity period of the data will be compared again. If the remaining validity period of data a is 2 seconds and the remaining validity period of data b is 10 seconds, then data a will be eliminated first.

[0093] Based on the above steps, by setting a reasonable elimination priority, the data to be eliminated can be intelligently selected according to the actual situation of the cache. For data with low popularity or large space, it is deleted from the cache first to ensure that the cache space is occupied by data with high access frequency, thereby improving the utilization rate of cache resources and the cache hit rate.

[0094] It should be understood that although Figure 3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0095] In one embodiment, Figure 8As shown, a cache data management device is provided, including: an evaluation module, a cache module, a diffusion module and a deletion module, wherein:

[0096] An evaluation module, used to count the access request frequency of each data within a unit time, and evaluate the access popularity of the data according to the access request frequency;

[0097] A cache module, configured to establish a cache of the target data in response to receiving an access request for the target data and the access request frequency of the target data reaching a preset cache threshold;

[0098] A diffusion module, used to determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0099] The deletion module is used for stopping diffusion of the target data and / or deleting the cache of the target data in response to the target data satisfying a preset expiration condition.

[0100] For the specific definition of the cache data management device, please refer to the definition of the cache data management method above, which will not be repeated here. Each module in the above-mentioned cache data management device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0101] In one embodiment, the diffusion module is also used to, in response to the access request frequency of the target data being less than a first frequency threshold, save the cache of the target data in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, match the target data with candidate nodes, determine at least one target diffusion node, and diffuse the target data to the cache service of the target diffusion node.

[0102] In one embodiment, the cache module is further used to compare the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, the cache of the target data is not saved and the database is directly accessed.

[0103] In one embodiment, the diffusion module is also used to collect resource usage parameters of each node and calculate the resource utilization rate of each node; screen the nodes whose resource utilization rate is less than a preset load threshold and add them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold and less than the second frequency threshold, randomly select at least one first target diffusion node from the candidate node list; or, record the cache hit status of each node and calculate the cache hit rate of the node; calculate the cache gain of all candidate nodes in the candidate node list according to the access popularity of the target data, the resource utilization rate of the node and the cache hit rate; obtain the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly select multiple second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtain the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0104] In one embodiment, the deletion module is also used to periodically scan the entire cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtain the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, delete the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, do not delete the cache of the target data, and determine whether to stop spreading the target data based on the current access heat.

[0105] In one embodiment, the deletion module is also used to stop the diffusion of the target data and delete the target data from the cache of the first target diffusion node in response to the current access request frequency being less than the first frequency threshold; in response to the current access request frequency being less than the second frequency threshold and greater than or equal to the first frequency threshold, delete the target data from the cache of the second target diffusion node, confirm the first target diffusion node based on the current access request frequency, and diffuse the target data to the cache service of the first target node.

[0106] In one embodiment, the deletion module is also used to obtain the data access heat, data size and remaining data validity period of the cached data in response to the amount of cached data reaching an upper limit threshold; set the elimination priority of the cached data according to the data access heat, data size and remaining data validity period; determine the target elimination data according to the elimination priority, and delete the target elimination data from the cache.

[0107] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store cache data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a cache data management method is implemented.

[0108] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0110] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency;

[0111] In response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data;

[0112] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0113] In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data.

[0114] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is stored in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0115] In one embodiment, when the processor executes the computer program, the following steps are also implemented: comparing the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, not saving the cache of the target data and directly accessing the database.

[0116] In one embodiment, when the processor executes the computer program, the following steps are also implemented: collecting resource usage parameters of each node and calculating the resource utilization rate of each node; screening nodes whose resource utilization rate is less than a preset load threshold and adding them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold and less than the second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording the cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list according to the access heat of the target data, the resource utilization rate of the node and the cache hit rate; obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0117] In one embodiment, when the processor executes the computer program, the following steps are also implemented: periodically scanning the entire cache data to determine whether the cache data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0118] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the current access request frequency being less than a first frequency threshold, the target data is stopped from being diffused, and the target data is deleted from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold and greater than or equal to the first frequency threshold, the target data is deleted from the cache of the second target diffusion node, and the first target diffusion node is confirmed according to the current access request frequency, and the target data is diffused to the cache service of the first target node.

[0119] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the amount of cached data reaching an upper limit threshold, the data access heat, data size and remaining data validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining data validity period; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0120] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0121] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency;

[0122] In response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data;

[0123] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0124] In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data.

[0125] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is stored in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0126] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: comparing the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, not saving the cache of the target data and directly accessing the database.

[0127] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: collecting resource usage parameters of each node and calculating the resource utilization rate of each node; screening nodes whose resource utilization rate is less than a preset load threshold and adding them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold and less than the second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording the cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list according to the access heat of the target data, the resource utilization rate of the node and the cache hit rate; obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0128] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: periodically scanning all cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold and greater than or equal to the first frequency threshold, deleting the target data from the cache of the second target diffusion node, confirming the first target diffusion node based on the current access request frequency, and diffusing the target data to the cache service of the first target node.

[0130] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the amount of cached data reaching an upper limit threshold, the data access heat, data size and remaining data validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining data validity period; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0131] In one embodiment, a computer product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0132] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency;

[0133] In response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data;

[0134] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0135] In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data.

[0136] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is stored in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0137] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: comparing the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, not saving the cache of the target data and directly accessing the database.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: collecting resource usage parameters of each node and calculating the resource utilization rate of each node; screening nodes whose resource utilization rate is less than a preset load threshold and adding them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold and less than the second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording the cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list according to the access heat of the target data, the resource utilization rate of the node and the cache hit rate; obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0139] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: periodically scanning all cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0140] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold and greater than or equal to the first frequency threshold, deleting the target data from the cache of the second target diffusion node, confirming the first target diffusion node based on the current access request frequency, and diffusing the target data to the cache service of the first target node.

[0141] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the amount of cached data reaching an upper limit threshold, the data access heat, data size and remaining data validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining data validity period; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), diffusion DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), diffusion link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A cache data management method, characterized in that: include: Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency; In response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data; Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node; In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data.

2. The cache data management method according to claim 1, characterized in that: The step of determining a cache diffusion strategy according to the access popularity of the target data, determining a target diffusion node according to the cache diffusion strategy, and diffusing the target data to the cache service of the target diffusion node includes: In response to the access request frequency of the target data being less than a first frequency threshold, storing the cache of the target data in the current node cache service without spreading, wherein the first frequency threshold is greater than the preset cache threshold; In response to the access request frequency of the target data being greater than or equal to a first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

3. The cache data management method according to claim 1 or 2, characterized in that: The method further comprises: Comparing the access request frequency of the target data with a preset cache threshold; In response to the frequency of access requests to the target data being less than a preset cache threshold, the cache of the target data is not saved, and the database is directly accessed.

4. The cache data management method according to claim 2, characterized in that: In response to the access request frequency of the target data being greater than or equal to a first frequency threshold, matching the target data with candidate nodes to determine at least one target diffusion node includes: Collect resource usage parameters of each node and calculate the resource utilization rate of each node; Filter nodes whose resource utilization is less than a preset load threshold and add them to the candidate node list; In response to the access request frequency of the target data being greater than or equal to a first frequency threshold and less than a second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; Or, record the cache hit situation of each node and calculate the cache hit rate of the node; Calculating cache gains of all candidate nodes in the candidate node list according to the access popularity of the target data, the resource utilization rate of the node, and the cache hit rate; Acquire a candidate node whose cache gain is greater than a preset diffusion threshold as a first target diffusion node; In response to the access request frequency of the target data being greater than or equal to a second frequency threshold, randomly selecting a plurality of second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; Or, a candidate node whose cache gain is greater than a preset diffusion threshold is obtained as a second target diffusion node.

5. The cache data management method according to claim 1 or 4, characterized in that: In response to the target data satisfying a preset failure condition, stopping diffusion of the target data and / or deleting a cache of the target data includes: Periodically scan all cached data to determine whether the cached data has exceeded its expiration time; In response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; In response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; In response to the current access heat being greater than or equal to a preset cache threshold, the cache of the target data is not deleted, and it is determined whether to stop diffusion of the target data according to the current access heat.

6. The cache data management method according to claim 5, characterized in that: In response to the current access heat being greater than or equal to a preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data according to the current access heat, including: In response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node; In response to the current access request frequency being less than a second frequency threshold and greater than or equal to a first frequency threshold, the target data is deleted from the cache of the second target diffusion node, and the first target diffusion node is confirmed based on the current access request frequency, and the target data is diffused to the cache service of the first target node.

7. The cache data management method according to claim 1, characterized in that: The method further comprises: In response to the cache data amount reaching an upper threshold, obtaining data access heat, data size, and remaining validity period of the cache data; Setting the elimination priority of the cached data according to the data access popularity, data size and remaining validity period of the data; The target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

8. A cache data management device, characterized in that: The device comprises: An evaluation module, used to count the access request frequency of each data within a unit time, and evaluate the access popularity of the data according to the access request frequency; A cache module, configured to establish a cache of the target data in response to receiving an access request for the target data and the access request frequency of the target data reaching a preset cache threshold; A diffusion module, used to determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node; The deletion module is used for stopping diffusion of the target data and / or deleting the cache of the target data in response to the target data satisfying a preset expiration condition.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images