Cache data management method, device, computer equipment and storage medium

Abstract

The present application relates to a cache data management method, apparatus, computer equipment and storage medium. The method includes: counting the access request frequency of each data in a unit time, and evaluating the access heat of the data according to the access request frequency; in response to receiving an access request for target data, and the access request frequency of the target data reaches a preset cache threshold, establishing a cache for the target data; determining a cache diffusion strategy according to the access heat of the target data, and determining a target diffusion node according to the cache diffusion strategy, and diffusing the target data to the cache service of the target diffusion node; in response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting the cache of the target data. The present method can be used to determine whether to establish a cache and whether to perform cache diffusion according to the data access heat, thereby avoiding low-frequency data occupying cache space.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a cache data management method, apparatus, computer equipment, and storage medium. Background Art

[0002] In modern web systems, to improve data access speed and reduce the pressure on the back-end database, a cache layer is usually designed between the web layer and the database layer. This design allows the web layer to obtain data from the cache layer first when querying data. Only when the cache misses will it access the database, update the data into the cache, and return the result.

[0003] There are two common architectures for Web systems that support high availability. The first architecture is Figure 1 As shown, each node's web layer and cache layer are independent, and the databases form a master-slave cluster, allowing each node to provide external services. In this architecture, if data does not exist in the cache, the system accesses the database and stores it in the cache. However, data is cached only after being accessed once. This can result in a large amount of infrequently accessed data in the cache, occupying a large amount of space and wasting resources. Furthermore, because the cache layers of each node are independent, users may repeatedly query data and update the cache when accessing different nodes, affecting interface access speed.

[0004] The second architecture is Figure 2 As shown in the figure, the cache layer of each node forms a distributed cache cluster. In this design, after a user visits any node, the data is stored in the distributed cache. When the user visits another node again, the data can be directly obtained from the distributed cache without having to re-cache, thus improving access speed. However, the cache layer of each node may occupy the same memory space. Even if some nodes do not receive any requests, their cache layers may still store a large amount of data, resulting in a waste of memory resources. Summary of the Invention

[0005] Based on this, it is necessary to provide a cache data management method, device, computer equipment and storage medium that can realize adaptive diffusion of cache data in response to the above technical problems.

[0006] In one aspect, a cache data management method is provided, the method comprising:

[0007] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data based on the access request frequency;

[0008] In response to receiving an access request for target data and a frequency of access requests for the target data reaching a preset cache threshold, establishing a cache for the target data;

[0009] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0010] In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data is performed.

[0011] In one embodiment, determining a cache diffusion strategy according to the access popularity of the target data, determining a target diffusion node according to the cache diffusion strategy, and diffusing the target data to a cache service of the target diffusion node includes:

[0012] In response to a frequency of access requests to the target data being less than a first frequency threshold, storing the cache of the target data in a current node cache service without spreading the cache, wherein the first frequency threshold is greater than the preset cache threshold;

[0013] In response to the access request frequency of the target data being greater than or equal to a first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0014] In one embodiment, the method further comprises:

[0015] Comparing the access request frequency of the target data with a preset cache threshold;

[0016] In response to the frequency of access requests to the target data being less than a preset cache threshold, the cache of the target data is not saved, and the database is directly accessed.

[0017] In one embodiment, in response to the frequency of access requests to the target data being greater than or equal to a first frequency threshold, matching the target data with candidate nodes to determine at least one target diffusion node includes:

[0018] Collect resource usage parameters of each node and calculate the resource utilization rate of each node;

[0019] Filter nodes whose resource utilization is less than a preset load threshold and add them to the candidate node list;

[0020] In response to a frequency of access requests to the target data being greater than or equal to a first frequency threshold and less than a second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list;

[0021] Or, record the cache hit situation of each node and calculate the cache hit rate of the node;

[0022] Calculating the cache gain of all candidate nodes in the candidate node list according to the access popularity of the target data, the resource utilization rate of the node, and the cache hit rate;

[0023] Acquire a candidate node whose cache gain is greater than a preset diffusion threshold as a first target diffusion node;

[0024] In response to a frequency of access requests to the target data being greater than or equal to a second frequency threshold, randomly selecting a plurality of second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes;

[0025] Alternatively, a candidate node whose cache gain is greater than a preset diffusion threshold is obtained as the second target diffusion node.

[0026] In one embodiment, in response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data includes:

[0027] Periodically scan all cached data to determine whether the cached data has exceeded its expiration time;

[0028] In response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data;

[0029] In response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache;

[0030] In response to the current access heat being greater than or equal to a preset cache threshold, the cache of the target data is not deleted, and whether to stop diffusion of the target data is determined according to the current access heat.

[0031] In one embodiment, in response to the current access popularity being greater than or equal to a preset cache threshold, not deleting the cache of the target data, and determining whether to stop spreading the target data based on the current access popularity, includes:

[0032] In response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node;

[0033] In response to the current access request frequency being less than a second frequency threshold and greater than or equal to a first frequency threshold, the target data is deleted from the cache of the second target diffusion node, and the first target diffusion node is confirmed based on the current access request frequency, and the target data is diffused to the cache service of the first target node.

[0034] In one embodiment, the method further comprises:

[0035] In response to the cached data amount reaching an upper threshold, obtaining data access popularity, data size, and remaining validity period of the cached data;

[0036] Setting the elimination priority of the cached data according to the data access popularity, data size and remaining validity period of the data;

[0037] Target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0038] In another aspect, a cache data management device is provided, the device comprising:

[0039] An evaluation module, configured to count the access request frequencies of each data item within a unit of time, and evaluate the access popularity of the data item based on the access request frequencies;

[0040] a cache module, configured to establish a cache for the target data in response to receiving an access request for the target data and when a frequency of access requests for the target data reaches a preset cache threshold;

[0041] a diffusion module, configured to determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0042] The deletion module is configured to stop spreading the target data and / or delete the cache of the target data in response to the target data satisfying a preset expiration condition.

[0043] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0044] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data based on the access request frequency;

[0045] In response to receiving an access request for target data and a frequency of access requests for the target data reaching a preset cache threshold, establishing a cache for the target data;

[0046] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0047] In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data is performed.

[0048] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0049] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data based on the access request frequency;

[0050] In response to receiving an access request for target data and a frequency of access requests for the target data reaching a preset cache threshold, establishing a cache for the target data;

[0051] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0052] In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data is performed.

[0053] The cache data management method, apparatus, computer device, and storage medium described above calculate the access request frequency of each data item per unit time and assess its popularity based on this frequency. This determines whether to establish a cache and whether to perform cache diffusion based on this popularity. This ensures that only frequently accessed data is cached, preventing low-frequency data from occupying cache space. This speeds up data access for other nodes and reduces database pressure. Furthermore, by stopping diffusion or deleting the cache based on preset expiration conditions, cache resources are released promptly, avoiding resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of a local cache architecture in the prior art;

[0055] Figure 2 A schematic diagram of a distributed cache architecture in the prior art;

[0056] Figure 3 1 is a flow chart of a cache data management method according to an embodiment;

[0057] Figure 4 FIG1 is a flow chart of steps for obtaining cold data in one embodiment;

[0058] Figure 5 A schematic diagram of a process flow for caching warm data in one embodiment;

[0059] Figure 6 A schematic diagram of a process flow for caching and spreading hot data in one embodiment;

[0060] Figure 7 FIG1 is a flow chart of the cache diffusion steps for extremely hot data in one embodiment;

[0061] Figure 8 is a structural block diagram of a cache data management device in one embodiment;

[0062] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] In one embodiment, Figure 3-7 As shown, a cache data management method is provided, comprising the following steps:

[0065] Step S1 : Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data according to the access request frequency.

[0066] Specifically, an access statistics aspect is implemented in the web layer. This aspect intercepts method calls in the web layer and uses a counter or sliding window algorithm to record the number of calls of each intercepted method per unit time, thereby obtaining the access request frequency of each data. And because it is implemented by the aspect, there is no need to modify the existing code logic.

[0067] For example, data access popularity is assessed based on the statistical access request frequency n. If n is less than 1 access request per second, the data is considered cold; if n is between 1 and 100 access requests per second, the data is considered warm; if n is between 100 and 300 access requests per second, the data is considered hot; and if n is greater than 300 access requests per second, the data is considered extremely hot. Subsequently, a corresponding cache diffusion strategy can be selected based on the data's access popularity, providing support for cache invalidation.

[0068] Step S2: in response to receiving an access request for target data and the frequency of the access request for the target data reaching a preset cache threshold, establishing a cache for the target data.

[0069] In one possible implementation, the access request frequency of the target data is compared with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, the cache of the target data is not saved, and the database is directly accessed.

[0070] Specifically, when an access request for target data is received, the access popularity of the data is first checked. If the access popularity is lower than the preset cache threshold, the database is directly accessed to obtain the data; if the access popularity exceeds the cache threshold, the regular cache process is followed to check whether the target data exists in the cache. If so, the data is returned directly from the cache. If not, the target data is obtained from the database and saved in the cache of the current node.

[0071] For example, Figure 4 As shown in the figure, the cache threshold is set to 1 time / second. When the access request frequency of the target data is less than 1 time / second, that is, the access heat of the target data is cold, the database can fully support queries of this frequency and will not save the data (method return value) to the cache. Therefore, no cache is established and the database is accessed directly.

[0072] like Figure 5 As shown in the figure, when the access request frequency of the target data is in the range of 1-100 times / second, that is, the access heat of the target data is warm, the database will face greater access pressure and a cache needs to be established. Therefore, the target data is saved in the current node cache service in key / value format, where the key is the called method name + method input parameters, and the value is the method return value. Subsequent access is directly obtained from the cache, thereby reducing database pressure.

[0073] Based on the above steps, whether to establish a cache is determined according to the data access popularity. If the data access frequency is low, no cache will be established. Since the access popularity of most data in the system is low-frequency data, it will not be saved in the cache. Therefore, the cache space occupied by low-frequency data can be reduced, thereby effectively utilizing limited cache resources and ensuring that data with high access frequency receives a quick response.

[0074] Step S3: determining a cache diffusion strategy according to the access popularity of the target data, determining a target diffusion node according to the cache diffusion strategy, and diffusing the target data to the cache service of the target diffusion node.

[0075] In one possible implementation, the cache diffusion strategy is determined based on the access popularity of the target data, and the target diffusion node is determined based on the cache diffusion strategy. The specific steps of diffusing the target data to the cache service of the target diffusion node include: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is saved in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0076] Among them, the preset cache threshold (for example, 1 time / second) is used to distinguish between cold data and hot data, thereby determining whether a cache needs to be established, and the first frequency threshold (for example, 100 times / second) is used to determine whether the data needs to be spread to other nodes. When the access request frequency of the target data is greater than or equal to the first frequency threshold, for example, 100 times / second, the access heat of the data is hot or extremely hot. Since the data request is very frequent, other nodes are likely to use the data. Therefore, in addition to establishing a cache, the data needs to be spread to the cache of other nodes so that when accessing other nodes is requested, the data can be directly obtained from the cache, thereby improving the response speed and reducing the pressure on the database.

[0077] Based on the above steps, frequently requested data is automatically diffused and a cache is established in advance. When other nodes receive a request for the data, they can directly obtain the data from the local cache without sending a query request to the database, ensuring that data with high access frequency receives a quick response and speeding up data access for other nodes.

[0078] In one embodiment, in response to the frequency of access requests to the target data being greater than or equal to a first frequency threshold, matching the target data with candidate nodes and determining at least one target diffusion node include: collecting resource usage parameters of each node and calculating the resource utilization of each node; screening nodes whose resource utilization is less than a preset load threshold and adding them to a candidate node list; in response to the frequency of access requests to the target data being greater than or equal to the first frequency threshold and less than a second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list based on the access popularity of the target data, the resource utilization of the node, and the cache hit rate; obtaining a candidate node whose cache gain is greater than a preset diffusion threshold as the first target diffusion node; in response to the frequency of access requests to the target data being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes from the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining a candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0079] Specifically, it is possible to obtain indicator information of the target data and determine the data capacity of the target data to be diffused based on the indicator information, the indicator information including the number of data requests, data size, and data bandwidth; obtain load information of each node, the load information including the equipment construction capacity and current carrying capacity of the cache device in the node; predict the expected carrying capacity of each node after the cache data of the target data is added based on the data capacity to be diffused and the current carrying capacity; determine the capacity utilization of each node after the target data is added based on the expected carrying capacity and the equipment construction capacity; traverse all nodes, determine the nodes whose capacity utilization is less than the load threshold as candidate nodes, and establish a list of candidate nodes.

[0080] For example, Figure 6 As shown, the cluster nodes include Node 1, Node 2, and Node 3. Assuming that these nodes all meet the resource utilization conditions and the second frequency threshold is set to 300 times / second, when Node 1 receives a request, if the target data is in the range of 1-300 times / second, that is, the access popularity is hot, a node can be randomly selected from Node 2 and Node 3, for example, Node 2, and then the data can be synchronized to the Node 2 cache service. The cache gain of Node 2 and Node 3 can also be calculated separately, and the node with the maximum cache gain can be selected. At this time, the Node 2 cache service can also provide external services. If a request accesses Node 2, the data can be directly obtained from the cache, improving the response speed and reducing the pressure on the database.

[0081] like Figure 7 As shown in the figure, if the access request frequency of the target data is greater than 300 times / second, that is, the access popularity is extremely got, cache diffusion is also performed, but more nodes will be selected to synchronize the data to the cache services of node 2 and node 3 to further distribute the load.

[0082] It should be noted that the number of target diffusion nodes can be determined based on the total number of cluster nodes. For example, if the cluster has 10 nodes, for data items with a hot access rate (100-300 times / second), 10% to 20% of the nodes can be selected as the first target diffusion nodes; for data items with an extremely hot access rate (greater than or equal to 300 times / second), 30% to 50% of the nodes can be selected as the second target diffusion nodes. Alternatively, by setting a diffusion threshold, all nodes whose calculated cache gain is greater than the diffusion threshold are selected as target diffusion nodes. In this case, for data items with an extremely hot access rate, the calculated cache gain is naturally greater than that of data items with a hot access rate. Therefore, when selecting nodes by calculating the cache gain, the number of second target diffusion nodes is still greater than the number of first target diffusion nodes.

[0083] Based on the above steps, by screening nodes with low resource utilization and establishing a candidate node list, we can fully utilize idle resources. The number of target diffusion nodes is determined by access popularity or calculating cache gain, thereby diffusing high-frequency data to the cache services of multiple nodes, ensuring that high-frequency data receives a quick response while reducing the pressure on the database.

[0084] Step S4: in response to the target data satisfying a preset expiration condition, stopping diffusion of the target data and / or deleting the cache of the target data.

[0085] In one possible implementation, in response to the target data satisfying a preset expiration condition, the specific steps of stopping diffusion of the target data and / or deleting the cache of the target data include: periodically scanning all cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0086] Specifically, to prevent data from occupying cache space all the time, a cache expiration mechanism has been designed. That is, each data is set with an expiration time when it is added to the cache. At the same time, a cache scanning thread is introduced. This thread periodically (for example, every 5 seconds) scans all cached data. If the scanning thread finds that the data has expired and the data popularity has decreased, the cache invalidation mechanism will be triggered to delete the cache of low-frequency data to reduce space usage. If a cached data is found to have expired, but the cached data popularity is warm or above, the data will not be deleted from the cache this time, and the cached data expiration time will be automatically extended, which is equivalent to exempting the expired data from this scan.

[0087] Based on the above steps, by periodically scanning all cached data and checking whether the data has exceeded the expiration time, the cache space occupied by data that is no longer frequently used can be released in a timely manner. For data that is still highly accessed, its cached data will be temporarily retained even if it has exceeded the preset expiration time, thereby improving the cache hit rate and reducing the pressure on the database.

[0088] In one possible implementation, in response to the current access heat being greater than or equal to a preset cache threshold, the cache of the target data is not deleted, and the specific steps of determining whether to stop spreading the target data based on the current access heat include: in response to the current access request frequency being less than a first frequency threshold, stopping spreading the target data and deleting the target data from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold and greater than or equal to the first frequency threshold, deleting the target data from the cache of the second target diffusion node, confirming the first target diffusion node based on the current access request frequency, and spreading the target data to the cache service of the first target node.

[0089] For example, if the data access heat decreases to warm, node 1 will no longer diffuse cache information to the outside. Since node 2 no longer receives cache data diffused by node 1, the data will automatically become invalid when the expiration time comes and will be deleted from node 2's cache. If the data heat decreases to hot, node 1 will reduce the diffusion of cache data to nodes 2 and 3 to only one node (such as node 2). Since node 3 no longer receives cache data diffused by node 1, the data will automatically become invalid when the expiration time comes and will be deleted from node 3's cache.

[0090] Based on the above steps, the introduction of a cache invalidation mechanism can decide whether to stop cache diffusion or delete cached data according to changes in data access popularity, and adjust the diffusion range of cached data. This can better adapt to changes in data access patterns and improve cache hit rates and system response speed.

[0091] In one possible implementation, in response to the amount of cached data reaching an upper threshold, the data access heat, data size and remaining validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining validity period of the data; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0092] Specifically, cached data is eliminated step by step based on access popularity, size, and remaining lifespan. For example, if the cache contains warm, hot, and extremely hot data, and cache capacity is insufficient, warm data will be eliminated first. If space is still insufficient, hot data will be eliminated next, and finally extremely hot data will be eliminated until the cached data volume falls below the upper threshold. When eliminating data of the same access popularity level, data with the largest space usage is prioritized. For example, when eliminating warm data, there are five warm data items, a, b, c, d, and e, occupying 20k, 2k, 3k, 4k, and 5k bytes, respectively. Data a is preferred because eliminating a alone frees up 20k bytes of space, while eliminating b, c, d, and e only frees up 14k bytes combined. Furthermore, eliminating one item has less impact than eliminating four items, making eliminating data a the most cost-effective. If multiple items are of the same size, the remaining lifespans of the items are compared. If data a has a remaining lifespan of 2 seconds and data b has a remaining lifespan of 10 seconds, data a will be eliminated first.

[0093] Based on the above steps, by setting a reasonable elimination priority, you can intelligently select the data to be eliminated according to the actual situation of the cache. For data with low popularity or large space, it will be deleted from the cache first to ensure that the cache space is occupied by data with high access frequency, thereby improving the utilization of cache resources and the cache hit rate.

[0094] It should be understood that although Figure 3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0095] In one embodiment, Figure 8As shown, a cache data management device is provided, including: an evaluation module, a cache module, a diffusion module and a deletion module, wherein:

[0096] An evaluation module, configured to count the access request frequencies of each data item within a unit of time, and evaluate the access popularity of the data item based on the access request frequencies;

[0097] a cache module, configured to establish a cache for the target data in response to receiving an access request for the target data and when a frequency of access requests for the target data reaches a preset cache threshold;

[0098] a diffusion module, configured to determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0099] The deletion module is configured to stop spreading the target data and / or delete the cache of the target data in response to the target data satisfying a preset expiration condition.

[0100] For the specific definition of the cache data management device, please refer to the definition of the cache data management method above, which will not be repeated here. The various modules in the above-mentioned cache data management device can be implemented in whole or in part by software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0101] In one embodiment, the diffusion module is also used to, in response to the access request frequency of the target data being less than a first frequency threshold, save the cache of the target data in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, match the target data with candidate nodes, determine at least one target diffusion node, and diffuse the target data to the cache service of the target diffusion node.

[0102] In one embodiment, the cache module is further configured to compare the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, the cache of the target data is not saved and the database is directly accessed.

[0103] In one embodiment, the diffusion module is further used to collect resource usage parameters of each node and calculate the resource utilization of each node; screen the nodes whose resource utilization is less than a preset load threshold and add them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold and less than the second frequency threshold, randomly select at least one first target diffusion node from the candidate node list; or, record the cache hit status of each node and calculate the cache hit rate of the node; calculate the cache gain of all candidate nodes in the candidate node list based on the access popularity of the target data, the resource utilization of the node and the cache hit rate; obtain the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly select multiple second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtain the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0104] In one embodiment, the deletion module is also used to periodically scan the entire cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtain the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, delete the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, do not delete the cache of the target data, and determine whether to stop diffusing the target data based on the current access heat.

[0105] In one embodiment, the deletion module is also used to stop the diffusion of the target data and delete the target data from the cache of the first target diffusion node in response to the current access request frequency being less than the first frequency threshold; in response to the current access request frequency being less than the second frequency threshold and greater than or equal to the first frequency threshold, delete the target data from the cache of the second target diffusion node, confirm the first target diffusion node based on the current access request frequency, and diffuse the target data to the cache service of the first target node.

[0106] In one embodiment, the deletion module is also used to obtain the data access heat, data size and remaining validity period of the cached data in response to the amount of cached data reaching an upper limit threshold; set the elimination priority of the cached data according to the data access heat, data size and remaining validity period of the data; determine the target elimination data according to the elimination priority, and delete the target elimination data from the cache.

[0107] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store cached data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a cached data management method is implemented.

[0108] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0109] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0110] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data based on the access request frequency;

[0111] In response to receiving an access request for target data and a frequency of access requests for the target data reaching a preset cache threshold, establishing a cache for the target data;

[0112] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0113] In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data is performed.

[0114] In one embodiment, when the processor executes the computer program, the following steps are further implemented: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is stored in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0115] In one embodiment, when the processor executes the computer program, it further implements the following steps: comparing the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, not saving the cache of the target data and directly accessing the database.

[0116] In one embodiment, when the processor executes the computer program, the following steps are further implemented: collecting resource usage parameters of each node and calculating the resource utilization of each node; screening nodes whose resource utilization is less than a preset load threshold and adding them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to a first frequency threshold and less than a second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording the cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list based on the access popularity of the target data, the resource utilization of the node and the cache hit rate; obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0117] In one embodiment, when the processor executes the computer program, it also implements the following steps: periodically scanning the entire cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0118] In one embodiment, when the processor executes the computer program, it further implements the following steps: in response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold and greater than or equal to the first frequency threshold, deleting the target data from the cache of the second target diffusion node, confirming the first target diffusion node based on the current access request frequency, and diffusing the target data to the cache service of the first target node.

[0119] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the amount of cached data reaching an upper limit threshold, the data access heat, data size and remaining validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining validity period of the data; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0120] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0121] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data based on the access request frequency;

[0122] In response to receiving an access request for target data and a frequency of access requests for the target data reaching a preset cache threshold, establishing a cache for the target data;

[0123] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0124] In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data is performed.

[0125] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is stored in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0126] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: comparing the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, not saving the cache of the target data and directly accessing the database.

[0127] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: collecting resource usage parameters of each node and calculating the resource utilization of each node; screening nodes whose resource utilization is less than a preset load threshold and adding them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to a first frequency threshold and less than a second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording the cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list based on the access popularity of the target data, the resource utilization of the node and the cache hit rate; obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes from the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0128] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: periodically scanning the entire cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: in response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold and greater than or equal to the first frequency threshold, deleting the target data from the cache of the second target diffusion node, confirming the first target diffusion node based on the current access request frequency, and diffusing the target data to the cache service of the first target node.

[0130] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the amount of cached data reaching an upper threshold, the data access heat, data size and remaining validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining validity period of the data; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0131] In one embodiment, a computer product is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0132] Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data based on the access request frequency;

[0133] In response to receiving an access request for target data and a frequency of access requests for the target data reaching a preset cache threshold, establishing a cache for the target data;

[0134] Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node;

[0135] In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data is performed.

[0136] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: in response to the access request frequency of the target data being less than a first frequency threshold, the cache of the target data is stored in the current node cache service without diffusion, wherein the first frequency threshold is greater than the preset cache threshold; in response to the access request frequency of the target data being greater than or equal to the first frequency threshold, the target data is matched with candidate nodes, at least one target diffusion node is determined, and the target data is diffused to the cache service of the target diffusion node.

[0137] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: comparing the access request frequency of the target data with a preset cache threshold; in response to the access request frequency of the target data being less than the preset cache threshold, not saving the cache of the target data and directly accessing the database.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: collecting resource usage parameters of each node and calculating the resource utilization of each node; screening nodes whose resource utilization is less than a preset load threshold and adding them to the candidate node list; in response to the access request frequency of the target data being greater than or equal to a first frequency threshold and less than a second frequency threshold, randomly selecting at least one first target diffusion node from the candidate node list; or, recording the cache hit status of each node and calculating the cache hit rate of the node; calculating the cache gain of all candidate nodes in the candidate node list based on the access popularity of the target data, the resource utilization of the node and the cache hit rate; obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the first target diffusion node; in response to the access request frequency of the target data being greater than or equal to the second frequency threshold, randomly selecting multiple second target diffusion nodes from the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, obtaining the candidate node whose cache gain is greater than the preset diffusion threshold as the second target diffusion node.

[0139] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: periodically scanning the entire cached data to determine whether the cached data exceeds the expiration time; in response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; in response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; in response to the current access heat being greater than or equal to the preset cache threshold, not deleting the cache of the target data, and determining whether to stop diffusion of the target data based on the current access heat.

[0140] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: in response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node; in response to the current access request frequency being less than a second frequency threshold and greater than or equal to the first frequency threshold, deleting the target data from the cache of the second target diffusion node, confirming the first target diffusion node based on the current access request frequency, and diffusing the target data to the cache service of the first target node.

[0141] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the amount of cached data reaching an upper threshold, the data access heat, data size and remaining validity period of the cached data are obtained; the elimination priority of the cached data is set according to the data access heat, data size and remaining validity period of the data; the target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

[0142] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), diffuse DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), diffuse link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0143] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A cache data management method, characterized in that: include: Counting the access request frequency of each data within a unit time, and evaluating the access popularity of the data based on the access request frequency; In response to receiving an access request for target data and a frequency of access requests for the target data reaching a preset cache threshold, establishing a cache for the target data; Determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node; In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data; The step of determining a cache diffusion strategy according to the access popularity of the target data and determining a target diffusion node according to the cache diffusion strategy includes: In response to a frequency of access requests to the target data being less than a first frequency threshold, storing the cache of the target data in a current node cache service without spreading the cache, wherein the first frequency threshold is greater than the preset cache threshold; In response to the frequency of access requests to the target data being greater than or equal to a first frequency threshold, matching the target data with candidate nodes to determine at least one target diffusion node includes: Collect resource usage parameters of each node and calculate the resource utilization rate of each node; Filter nodes whose resource utilization is less than a preset load threshold and add them to the candidate node list; In response to the frequency of access requests for the target data being greater than or equal to a first frequency threshold and less than a second frequency threshold, recording a cache hit status of each node and calculating a cache hit rate of the node; Calculating the cache gain of all candidate nodes in the candidate node list according to the access popularity of the target data, the resource utilization rate of the node, and the cache hit rate; Acquire a candidate node whose cache gain is greater than a preset diffusion threshold as a first target diffusion node; In response to the access request frequency of the target data being greater than or equal to a second frequency threshold, a plurality of second target diffusion nodes are randomly selected according to the candidate node list, wherein the number of the second target diffusion nodes is greater than the number of the first target diffusion nodes; or, a candidate node whose cache gain is greater than a preset diffusion threshold is obtained as the second target diffusion node.

2. The cache data management method according to claim 1, wherein: The method further comprises: Comparing the access request frequency of the target data with a preset cache threshold; In response to the frequency of access requests to the target data being less than a preset cache threshold, the cache of the target data is not saved, and the database is directly accessed.

3. The cache data management method according to claim 1, wherein: In response to the target data meeting a preset expiration condition, stopping diffusion of the target data and / or deleting a cache of the target data includes: Periodically scan all cached data to determine whether the cached data has exceeded its expiration time; In response to the cache time of the target data exceeding the expiration time, obtaining the current access request frequency of the target data; In response to the current access request frequency being less than a preset cache threshold, deleting the target data from the cache; In response to the current access request frequency being greater than or equal to a preset cache threshold, the cache of the target data is not deleted, and whether to stop diffusion of the target data is determined according to the current access request frequency.

4. The cache data management method according to claim 3, wherein: In response to the current access request frequency being greater than or equal to a preset cache threshold, not deleting the cache of the target data, and determining whether to stop spreading the target data according to the current access request frequency, including: In response to the current access request frequency being less than a first frequency threshold, stopping diffusion of the target data and deleting the target data from the cache of the first target diffusion node; In response to the current access request frequency being less than a second frequency threshold and greater than or equal to a first frequency threshold, the target data is deleted from the cache of the second target diffusion node, and the first target diffusion node is confirmed based on the current access request frequency, and the target data is diffused to the cache service of the first target diffusion node.

5. The cache data management method according to claim 1, wherein: The method further comprises: In response to the cached data amount reaching an upper threshold, obtaining data access popularity, data size, and remaining validity period of the cached data; Setting the elimination priority of the cached data according to the data access popularity, data size and remaining validity period of the data; Target elimination data is determined according to the elimination priority, and the target elimination data is deleted from the cache.

6. A cache data management device for implementing the cache data management method according to any one of claims 1 to 5, characterized in that: The device comprises: An evaluation module, configured to count the access request frequencies of each data item within a unit of time, and evaluate the access popularity of the data item based on the access request frequencies; a cache module, configured to establish a cache for the target data in response to receiving an access request for the target data and when a frequency of access requests for the target data reaches a preset cache threshold; a diffusion module, configured to determine a cache diffusion strategy according to the access popularity of the target data, determine a target diffusion node according to the cache diffusion strategy, and diffuse the target data to the cache service of the target diffusion node; The deletion module is configured to stop spreading the target data and / or delete the cache of the target data in response to the target data satisfying a preset expiration condition.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images