Local cache refreshing method and device, electronic equipment and readable medium

Abstract

The invention relates to a local cache refreshing method and device, electronic equipment and a readable medium, and the method comprises the steps: preloading common data to a local cache, the common data being obtained by analyzing historical log data; controlling the data volume of the local cache to be smaller than or equal to the capacity threshold value of the local cache through a first strategy, and dynamically adjusting the survival time of each data item in the local cache through a second strategy; and collecting a data change record of the local cache, asynchronously synchronizing the data change record to the server, and refreshing the local cache. Common data are preloaded to a local cache, then a first strategy and a second strategy are utilized to quantitatively control the data volume of the local cache and flexibly adjust the survival time of each data item, and finally a data change record is synchronized to a server and the local cache is refreshed. The problem that cache data processing is not flexible enough during cache management is solved.

Description

Technical Field

[0001] The present application relates to the field of cache processing technology, and in particular to a local cache refresh method, device, electronic device and readable medium. Background Art

[0002] In today's application scenarios that are highly dependent on network connections, timely synchronization and efficient access to data are crucial. However, when network connections are limited or unstable, data availability and access performance are often severely affected. Traditional cache refresh strategies, such as fixed refresh or TTL (Time-To-Live)-based refresh, can maintain the validity of cached data to a certain extent. However, in scenarios with high concurrency or frequent requests, fixed refresh and TTL-based refresh strategies are not flexible enough in handling cached data during cache management, and cannot be dynamically adjusted according to data access patterns, which can easily lead to problems such as bandwidth waste or low cache hit rate.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The present application provides a local cache refresh method, device, electronic device and readable medium to solve the above-mentioned technical problem of insufficient flexibility in processing cache data during cache management.

[0005] According to one aspect of an embodiment of the present application, the present application provides a method for refreshing a local cache, comprising: preloading commonly used data into the local cache, wherein the commonly used data is obtained by analyzing historical log data; controlling the amount of data in the local cache to be less than or equal to a capacity threshold of the local cache through a first strategy, and dynamically adjusting the lifetime of each data item in the local cache through a second strategy; collecting data change records of the local cache, and asynchronously synchronizing the data change records to a server, and refreshing the local cache.

[0006] Optionally, the first strategy is used to control the amount of data in the local cache to be less than or equal to a capacity threshold of the local cache, including: generating a hash table and a bidirectional linked list based on each data item in the local cache; upon receiving a data write request, obtaining the current amount of data in the local cache; if the current amount of data is equal to the capacity threshold of the local cache, removing the least recently used data item from the tail of the bidirectional linked list, and deleting the key-value pair corresponding to the least recently used data item in the hash table; when the current amount of data is less than the capacity threshold of the local cache, adding the new data item corresponding to the data write request to the head of the hash table and the bidirectional linked list.

[0007] Optionally, the lifetime of each data item in the local cache is dynamically adjusted through a second strategy, including: monitoring the access frequency and cache hit rate of each data item in the local cache; dynamically adjusting the initial lifetime of the data item based on the access frequency and cache hit rate, wherein the initial lifetime is the initial value set when the data item is added to the cache.

[0008] Optionally, after preloading the commonly used data into the local cache, the method further includes: monitoring the validity of each data item in the local cache; if a first data item is detected, preloading new data, wherein the first data item is a data item that is about to expire.

[0009] Optionally, before preloading the commonly used data into the local cache, the method also includes obtaining the commonly used data in the following manner: calculating the access frequency of each data item in the historical log data, and performing cluster analysis on the historical log data to obtain multiple clusters; identifying the access pattern of the data items in each cluster; and extracting the commonly used data from the historical log data based on the access frequency and access pattern.

[0010] Optionally, after dynamically adjusting the lifetime of each data item in the local cache through the second strategy, the method also includes: if a cache request issued by the target object is received, parsing the cache request to determine the second data item to be found; searching for the second data item in the local cache; if the second data item exists in the local cache and the lifetime of the second data item has not expired, it is determined to be a cache hit, if the second data item does not exist in the local cache, or the second data item exists in the local cache and the lifetime of the second data item has expired, it is determined to be a cache miss; in the case of a cache hit, reading the second data item, in the case of a cache miss, obtaining the second data item from the data source, and adding the second data item to the local cache according to the first strategy; sending the second data item to the target object.

[0011] Optionally, the data change records are asynchronously synchronized to the server, and the local cache is refreshed, including: when it is detected that the network connection status is stable, extracting the data change records in the task queue, wherein the task queue includes at least one collected data change record; generating a data request corresponding to the data change record, and sending the data request to the server; after receiving a response message from the server, refreshing the local cache, and clearing the processed data change records in the task queue.

[0012] According to another aspect of an embodiment of the present application, the present application provides a local cache refresh device, including: a loading module, used to preload commonly used data into the local cache, wherein the commonly used data is obtained by analyzing historical log data; a control module, used to control the amount of data in the local cache to be less than or equal to the capacity threshold of the local cache through a first strategy, and dynamically adjust the life span of each data item in the local cache through a second strategy; a synchronization module, used to collect data change records of the local cache, and asynchronously synchronize the data change records to the server, and refresh the local cache.

[0013] According to another aspect of an embodiment of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the steps of the above method are implemented when the processor executes the computer program.

[0014] According to another aspect of an embodiment of the present application, the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, and the program code enables the processor to execute the above method.

[0015] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:

[0016] The present application provides a method for refreshing a local cache, including: preloading commonly used data into the local cache, wherein the commonly used data is obtained by analyzing historical log data; controlling the amount of data in the local cache to be less than or equal to the capacity threshold of the local cache through a first strategy, and dynamically adjusting the life span of each data item in the local cache through a second strategy; collecting data change records in the local cache, and asynchronously synchronizing the data change records to the server, and refreshing the local cache. By first preloading commonly used data into the local cache, and then using the first strategy and the second strategy to quantitatively control the amount of data in the local cache and flexibly adjust the life span of each data item, and finally synchronizing the data change records to the server and refreshing the local cache, the problem of insufficient flexibility in handling cache data during cache management is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of an optional local cache refresh method provided according to an embodiment of the present application;

[0020] Figure 2 A flowchart of an optional cache management method provided according to an embodiment of the present application;

[0021] Figure 3 A flowchart of an optional historical log data analysis method provided according to an embodiment of the present application;

[0022] Figure 4 A flowchart of an optional data synchronization method provided according to an embodiment of the present application;

[0023] Figure 5 A schematic diagram of an optional local cache refresh provided according to an embodiment of the present application;

[0024] Figure 6 A block diagram of an optional local cache refresh device provided according to an embodiment of the present application;

[0025] Figure 7 A schematic diagram of an optional electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0028] In today's application scenarios that are highly dependent on network connections, timely synchronization and efficient access to data are crucial. However, when network connections are limited or unstable, data availability and access performance are often severely affected. Traditional cache refresh strategies, such as fixed refresh or TTL (Time-To-Live)-based refresh, can maintain the validity of cached data to a certain extent. However, in scenarios with high concurrency or frequent requests, fixed refresh and TTL-based refresh strategies are not flexible enough in handling cached data during cache management, and cannot be dynamically adjusted according to data access patterns, which can easily lead to problems such as bandwidth waste or low cache hit rate.

[0029] High concurrency: refers to the system's need to handle a large number of concurrent requests, that is, many users simultaneously access the same API (Application Programming Interface), URL (Uniform Resource Locator) or perform the same operation. This often occurs in business scenarios with a large number of active users and a high degree of user aggregation.

[0030] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiment of the present application, a method for refreshing a local cache is provided, such as Figure 1 As shown, including:

[0031] Step 101, preloading commonly used data into a local cache, wherein the commonly used data is obtained by analyzing historical log data;

[0032] Step 103, controlling the amount of data in the local cache to be less than or equal to the capacity threshold of the local cache through the first strategy, and dynamically adjusting the survival time of each data item in the local cache through the second strategy;

[0033] Step 105: collect data change records in the local cache, asynchronously synchronize the data change records to the server, and refresh the local cache.

[0034] This application first preloads commonly used data into the local cache, then controls the amount of data in the local cache to be less than or equal to the capacity threshold of the local cache through the first strategy, and dynamically adjusts the lifetime of each data item in the local cache through the second strategy, then monitors the data change records, and finally synchronizes the data change records and refreshes the local cache.

[0035] Specifically, the historical log data in the embodiments of the present application may be collected log data about data changes, and the log data may include information such as update time, update content, success or failure of the update operation involved in the data change process.

[0036] Specifically, the commonly used data in the embodiments of the present application can be understood as resource data with a high access frequency and predicted to have a large number of accesses in the future.

[0037] The local cache refresh method provided in the present application includes: when the cached data is outdated or no longer reliable, all data in the local cache is replaced with the latest data version; the contents of the local cache are resynchronized according to the status of the external data source, such as checking the changes in the data source and applying these changes to the local cache; and data items in the local cache that are expired or no longer needed are removed to free up storage space and maintain the validity of the cache.

[0038] The local cache refresh method provided in this application analyzes historical log data, uses machine learning algorithms to identify commonly used data and preloads it into the local cache, and introduces dynamic refresh rules in cache management to ensure data availability even in high concurrency or cache breakdown.

[0039] The cache management of data in the local cache provided by this application includes four parts, namely: preloading new data when the cache is about to expire; using the LRU (Least Recently Used) algorithm to manage the cache space; dynamically adjusting the survival time; and processing cache requests.

[0040] Figure 2 The method flow chart of cache management for data in the local cache provided for the present application, as shown in the figure, includes: loading commonly used data into the cache; checking the validity of the cache; if the cache is about to expire, preloading new data; applying the LRU strategy to dynamically manage the cache; adaptive TTL strategy adjustment; processing cache requests; if the cache hits, returning the cached data; if the cache misses, obtaining data from the server and updating the cache.

[0041] Next, based on Figure 2 Describes in detail how to manage the local cache.

[0042] This application adopts the LRU cache replacement strategy combined with the adaptive TTL strategy to adaptively manage cache data and start preloading when the cache is about to expire. It improves the cache space utilization, avoids cache pollution, and ensures data access performance, especially in high concurrency situations to avoid cache breakdown.

[0043] As an optional embodiment, the first strategy is used to control the amount of data in the local cache to be less than or equal to the capacity threshold of the local cache, including: generating a hash table and a bidirectional linked list based on each data item in the local cache; when a data write request is received, obtaining the current amount of data in the local cache; if the current amount of data is equal to the capacity threshold of the local cache, removing the longest unused data item from the tail of the bidirectional linked list, and deleting the key-value pair corresponding to the longest unused data item in the hash table; when the current amount of data is less than the capacity threshold of the local cache, adding the new data item corresponding to the data write request to the head of the hash table and the bidirectional linked list.

[0044] Since it is necessary to select a suitable data structure to store cache items and their access order, a hash table and a doubly linked list are generated according to each data item in the local cache. The hash table is used to quickly find cache items, and its key is the key of the cache item, and the value is the node reference in the doubly linked list; the doubly linked list is used to arrange the cache items in the order of access, with the most recently accessed items near the head of the linked list and the items that have not been accessed for the longest time near the tail of the linked list.

[0045] The capacity threshold of the local cache is predefined, which can be understood as the upper limit of the number of data items that can be stored in the local cache.

[0046] When a data write request is received, it means that a new data item needs to be written to the local cache. At this time, check whether the cache has reached the capacity limit (that is, the current data volume is less than the capacity threshold of the local cache); if it is not full (that is, the current data volume is less than the capacity threshold of the local cache), directly add the new data item to the head of the hash table and the doubly linked list. If it is full (that is, the current data volume is equal to the capacity threshold of the local cache), it is necessary to eliminate the data item that has not been used for the longest time, including: removing the data item that has not been used for the longest time from the tail of the doubly linked list, and deleting the corresponding key-value pair in the hash table, and then adding the new data item to the head of the hash table and the doubly linked list, indicating that the data item was recently written (or accessed).

[0047] In addition, for a data item that already exists in the cache (that is, a key that already exists in the hash table), if it is accessed again, its position will be updated to the head in the doubly linked list to reflect its most recently used status.

[0048] When adding, deleting, or accessing data items, the hash table and the doubly linked list need to be updated simultaneously to maintain data consistency between the two.

[0049] The LRU cache has an automatic elimination mechanism. When the cache reaches its capacity limit, it automatically removes the least recently used data items without manual intervention. This automated management method simplifies cache maintenance and reduces performance issues caused by cache overflow.

[0050] As an optional embodiment, the lifetime of each data item in the local cache is dynamically adjusted through a second strategy, including: monitoring the access frequency and cache hit rate of each data item in the local cache; dynamically adjusting the initial lifetime of the data item based on the access frequency and cache hit rate, wherein the initial lifetime is the initial value set when the data item is added to the cache.

[0051] The second strategy provided by the present application includes dynamically adjusting the initial life time of the data item according to the access frequency and / or the cache hit rate.

[0052] Continuously monitor the access status of each data item and calculate the access frequency of each data item; if the access frequency is higher than a first preset value, the data item is considered to be a high-frequency access data item, and the initial lifetime value of the data item can be increased; if the access frequency is lower than a second preset value, the data item is considered to be a low-frequency access data item, and the initial lifetime value of the data item is lowered, wherein the first preset value can be greater than the second preset value, and the present application does not limit the specific values ​​of the first preset value and the second preset value.

[0053] For frequently accessed data items, increase their time to live value so that they can remain in the cache longer. This can reduce the number of requests to the original server, increase the cache hit rate, and thus improve system performance.

[0054] For data items that are accessed infrequently, their time to live values ​​are reduced so that these data items can be updated faster in the cache. This ensures that the data in the cache remains fresh and prevents users from accessing outdated information.

[0055] The cache hit rate refers to the ratio of the number of successful data reads from the cache to the total number of reads. This ratio is one of the important indicators for measuring cache system performance and is usually expressed as a percentage. A higher hit rate means that the data stored in the cache is frequently accessed, while a lower hit rate may indicate that there is a large amount of infrequently used data in the cache. Therefore, a feedback mechanism can also be established to monitor and adjust cache performance through the cache hit rate: by collecting and analyzing key indicators such as cache hit rate and data access pattern, the survival time strategy (second strategy) is automatically adjusted to adapt to actual access needs.

[0056] By dynamically adjusting the lifetime and establishing a feedback mechanism, the cache system can more flexibly adapt to changes in data access patterns, thereby improving the cache hit rate and overall performance.

[0057] As an optional embodiment, after preloading the commonly used data into the local cache, the method further includes: monitoring the validity of each data item in the local cache; if a first data item is detected, preloading new data, wherein the first data item is a data item that is about to expire.

[0058] After selecting the frequently used data, the frequently used data is first loaded into the local cache. At this time, an initial survival time is set for each cache data item. The initial survival time of different data items can be determined according to the access frequency, importance or business rules of the data.

[0059] Specifically, the validity of each data item in the local cache is monitored, and new data is preloaded when the first data item appears, including: periodically checking the validity of each data item in the local cache (or checking when certain events (such as data updates) occur); checking the validity of the data includes: checking whether the lifetime of the data item has expired or is about to expire (for example, the lifetime drops to a preset threshold); when it is detected that the cache is about to expire (that is, the first data item appears), triggering the preloading logic to load new data from the data source, wherein the data source can be a database or an external service, etc.

[0060] It should be noted that the preloading of new data provided by the present application is performed asynchronously to avoid blocking the current user request. After the preloading is successful, the data items in the local cache are updated, including the value of the data item, the last access time and the new survival time.

[0061] As an optional embodiment, before preloading the commonly used data into the local cache, the method also includes obtaining the commonly used data in the following manner: calculating the access frequency of each data item in the historical log data, and performing cluster analysis on the historical log data to obtain multiple clusters; identifying the access pattern of the data items in each cluster; and extracting the commonly used data from the historical log data based on the access frequency and access pattern.

[0062] Before analyzing historical log data, a series of preprocessing is required, including data cleaning, data format conversion, etc. There are many mature processing methods available in the prior art, and this application does not limit the specific steps of data preprocessing.

[0063] Optionally, a weighted moving average algorithm is used to calculate the access frequency of each data item.

[0064] Optionally, a clustering algorithm is used to cluster and group the historical log data to determine multiple clusters; the data items in each cluster are analyzed to identify common features between them, such as access time, user group, resource type, etc., to identify access patterns.

[0065] Preferably, historical log data may be used as training data to train a prediction model to learn data access patterns and predict future access trends.

[0066] By combining the access frequency, access mode and predicted access trend of each data item, data items with high access frequency and expected large future access volume can be selected as common data. Common data may include multiple data items.

[0067] Figure 3 The analysis method of historical log data provided for this application, as shown in the figure, includes: collecting historical log data; analyzing log data: analyzing the access frequency and access pattern of each data item in the historical log data, and predicting the access trend; data validity detection: monitoring the validity of each data item; if the data is valid, identifying the commonly used data and generating a commonly used data list; if the data is invalid, recording an abnormal log; storing the commonly used data analysis results.

[0068] The weighted moving average algorithm, cluster analysis and prediction model are used to analyze historical logs, calculate data access frequency and access patterns, and introduce multiple algorithms for comprehensive analysis to extract user access habits and efficiently identify commonly used data.

[0069] The present application provides a method for refreshing a local cache, including: preloading commonly used data into the local cache, wherein the commonly used data is obtained by analyzing historical log data; controlling the amount of data in the local cache to be less than or equal to the capacity threshold of the local cache through a first strategy, and dynamically adjusting the life span of each data item in the local cache through a second strategy; collecting data change records in the local cache, and asynchronously synchronizing the data change records to the server, and refreshing the local cache. By first preloading commonly used data into the local cache, and then using the first strategy and the second strategy to quantitatively control the amount of data in the local cache and flexibly adjust the life span of each data item, and finally synchronizing the data change records to the server and refreshing the local cache, the problem of insufficient flexibility in handling cache data during cache management is solved.

[0070] As an optional embodiment, after dynamically adjusting the lifetime of each data item in the local cache through the second strategy, the method also includes: if a cache request issued by the target object is received, parsing the cache request to determine the second data item to be found; searching for the second data item in the local cache; if the second data item exists in the local cache and the lifetime of the second data item has not expired, it is determined to be a cache hit, if the second data item does not exist in the local cache, or the second data item exists in the local cache and the lifetime of the second data item has expired, it is determined to be a cache miss; in the case of a cache hit, reading the second data item, in the case of a cache miss, obtaining the second data item from the data source, and adding the second data item to the local cache according to the first strategy; sending the second data item to the target object.

[0071] When a cache request is received from a target object, the cache request needs to be parsed first, because the cache request usually contains some key information, such as the unique identifier of the data item to be found (such as ID, key name, etc.). After parsing the cache request, the second data item corresponding to the cache request can be determined.

[0072] The requested second data item is searched in the local cache, which can be specifically searched through a hash table. The hash table maps the input (such as the key or unique identifier of the data item) to a small, fixed array index through a mechanism called a hash function, and this index is used to quickly search, insert or delete data items in the hash table.

[0073] If the second data item cannot be found in the local cache according to the hash table, it is determined as a cache miss. If the second data item is found in the local cache according to the hash table, it is necessary to further verify whether the second data item is still valid. If it is valid, it is determined as a cache hit. If not, it is determined as a cache miss.

[0074] The expiration time is a pre-set value that indicates the length of time that a data item can remain valid in the cache. The expiration time can be used to verify whether the second data item is still valid: check whether the expiration time of the second data item has expired. If the expiration time of the second data item has not expired, it indicates that the second data item is still valid and can be used directly. If the expiration time of the second data item has expired, it indicates that the second data item is invalid.

[0075] If the cache hits, the second data item in the cache is directly read and returned to the target object.

[0076] If the cache misses, it is necessary to obtain the second data item from the data source, then add the second data item as new data to the local cache according to the first strategy mentioned above, and finally return the second data item to the target object.

[0077] As an optional embodiment, data change records are asynchronously synchronized to a server, and a local cache is refreshed, including: when it is detected that the network connection status is a stable connection, extracting the data change records in a task queue, wherein the task queue includes at least one collected data change record; generating a data request corresponding to the data change record, and sending the data request to the server; after receiving a response message from the server, refreshing the local cache, and clearing the processed data change records in the task queue.

[0078] Figure 4 A flowchart of the data synchronization method provided for the present application, as shown in the figure, the synchronization method includes: initializing an asynchronous task queue; collecting cache change records; adding change records to the queue; judging the network connection status; if the network connection is stable, executing the asynchronous data synchronization task; if the network connection is unstable, waiting for the network to recover and then executing the asynchronous data synchronization task; updating the local cache; and clearing the processed tasks in the synchronization queue.

[0079] Create a task queue for storing synchronization tasks to be executed. For example, create a thread-safe queue. Each element in the task queue can be an object containing task details, such as change records, target URLs, necessary request headers, etc.

[0080] Monitor changes in the local cache through cache change listeners or triggers. Whenever a change is detected, record the detailed information of the change (such as change type, data ID involved, new value, etc.), encapsulate the change record into a task object, and add it to the created asynchronous task queue.

[0081] Before trying to synchronize data, check whether the network connection is stable to avoid invalid data transmission when the network is unavailable or unstable. The specific steps include: using the operating system's network API or third-party library to check the network connection status; based on the network check results, decide whether to perform the synchronization task immediately or wait for the network to recover.

[0082] When the network connection is stable, the task extracts data change records from the queue, builds the corresponding data request, and sends the data to the server. If a response from the server (such as status code, response body, response header, etc.) is received, the response message is processed according to actual needs (such as error retry, update local status, etc.).

[0083] After the data is successfully synchronized to the server, the local cache is updated according to the server's response or local policy, including but not limited to marking synchronized data items, updating timestamps, or clearing old cached data.

[0084] After completing the update of the local cache, remove the tasks that have been successfully processed or are no longer needed in the task queue to avoid infinite queue growth. This ensures that data changes can be synchronized to the server efficiently and reliably, while reducing the waste of network resources and the impact on user experience.

[0085] This application uses an asynchronous task queue to efficiently manage synchronization tasks, and combines a differential synchronization strategy to reduce bandwidth consumption. The differential synchronization strategy is introduced to reduce unnecessary data transmission and improve synchronization efficiency.

[0086] The local cache refresh method provided in this application can be extended to multiple application scenarios, such as edge computing, cloud computing platforms, etc.

[0087] Figure 5 The schematic diagram of the refresh of the local cache provided for this application, as shown in the figure, the entire refresh process is carried out by 4 modules, namely the historical log analysis module, the cache loading module, the data synchronization module and the cache management module, wherein the historical log analysis module is used for log collection (including collecting log data) and log analysis (including analyzing logs and identifying commonly used data); the cache loading module is used for cache loading (including preloading commonly used data); the data synchronization module is used for data synchronization (including collecting cache updates and asynchronous synchronization); the cache management module is used for cache management (including managing cache and handling cache misses). The historical log analysis module will transmit the output common data analysis results to the cache loading module, the cache loading module will preload common data according to the common data analysis results, the cache management module will load common data for cache management, and the data synchronization module will obtain the cache change records in the cache management module and perform data synchronization.

[0088] This application proposes a method for intelligently preloading commonly used data into a local cache through historical logs, allowing efficient data synchronization between the server and the local cache, ensuring data availability even when network connections are limited or unstable, saving bandwidth and improving performance.

[0089] The present application provides a method for refreshing a local cache, including: preloading commonly used data into the local cache, wherein the commonly used data is obtained by analyzing historical log data; controlling the amount of data in the local cache to be less than or equal to the capacity threshold of the local cache through a first strategy, and dynamically adjusting the life span of each data item in the local cache through a second strategy; collecting data change records in the local cache, and asynchronously synchronizing the data change records to the server, and refreshing the local cache. By first preloading commonly used data into the local cache, and then using the first strategy and the second strategy to quantitatively control the amount of data in the local cache and flexibly adjust the life span of each data item, and finally synchronizing the data change records to the server and refreshing the local cache, the problem of insufficient flexibility in handling cache data during cache management is solved.

[0090] According to another aspect of the embodiment of the present application, the present application provides a local cache refresh device, such as Figure 6 As shown, including:

[0091] A loading module 602 is used to preload commonly used data into a local cache, wherein the commonly used data is obtained by analyzing historical log data;

[0092] A control module 604, configured to control the amount of data in the local cache to be less than or equal to a capacity threshold of the local cache through a first strategy, and to dynamically adjust the lifetime of each data item in the local cache through a second strategy;

[0093] The synchronization module 606 is used to collect data change records in the local cache, asynchronously synchronize the data change records to the server, and refresh the local cache.

[0094] It should be noted that the loading module 602 in this embodiment can be used to execute step 101 in the embodiment of the present application, the control module 604 in this embodiment can be used to execute step 103 in the embodiment of the present application, and the synchronization module 606 in this embodiment can be used to execute step 105 in the embodiment of the present application.

[0095] Optionally, the control module 604 is also specifically used to generate a hash table and a doubly linked list based on each data item in the local cache; when a data write request is received, the current data amount of the local cache is obtained; if the current data amount is equal to the capacity threshold of the local cache, the longest unused data item is removed from the tail of the doubly linked list, and the key-value pair corresponding to the longest unused data item is deleted from the hash table; when the current data amount is less than the capacity threshold of the local cache, the new data item corresponding to the data write request is added to the hash table and the head of the doubly linked list.

[0096] Optionally, the control module 604 is also specifically used to monitor the access frequency and cache hit rate of each data item in the local cache; dynamically adjust the initial survival time of the data item according to the access frequency and cache hit rate, wherein the initial survival time is the initial value set when the data item is added to the cache.

[0097] Optionally, the device also includes a first processing module, which is used to monitor the validity of each data item in the local cache after preloading the commonly used data into the local cache; if a first data item is detected, new data is preloaded, wherein the first data item is a data item that is about to expire.

[0098] Optionally, the device also includes a second processing module, which is used to obtain the commonly used data before preloading the commonly used data into the local cache, including in the following manner: calculating the access frequency of each data item in the historical log data, and performing cluster analysis on the historical log data to obtain multiple clusters; identifying the access pattern of the data items in each cluster; extracting the commonly used data from the historical log data through the access frequency and access pattern.

[0099] Optionally, the device also includes a third processing module, which is used to parse the cache request if a cache request from the target object is received after the lifetime of each data item in the local cache is dynamically adjusted through the second strategy, so as to determine the second data item to be found; search for the second data item in the local cache; if the second data item exists in the local cache and the lifetime of the second data item has not expired, it is determined to be a cache hit, if the second data item does not exist in the local cache, or the second data item exists in the local cache and the lifetime of the second data item has expired, it is determined to be a cache miss; in the case of a cache hit, read the second data item, in the case of a cache miss, obtain the second data item from the data source, and add the second data item to the local cache according to the first strategy; send the second data item to the target object.

[0100] Optionally, the synchronization module 606 is also used to extract the data change record in the task queue when it is detected that the network connection status is a stable connection, wherein the task queue includes at least one collected data change record; generate a data request corresponding to the data change record, and send the data request to the server; after receiving the server's response message, refresh the local cache and clear the processed data change record in the task queue.

[0101] It should be noted here that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments.

[0102] According to another aspect of the embodiments of the present application, the present application provides an electronic device, such as Figure 7As shown, it includes a memory 701, a processor 703, a communication interface 705 and a communication bus 707. The memory 701 stores a computer program that can be run on the processor 703. The memory 701 and the processor 703 communicate through the communication interface 705 and the communication bus 707. When the processor 703 executes the computer program, the steps of the above method are implemented.

[0103] The memory and processor in the above electronic device communicate via a communication bus and a communication interface. The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc.

[0104] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0105] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0106] According to another aspect of the embodiments of the present application, a computer-readable medium having a non-volatile program code executable by a processor is provided.

[0107] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0108] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.

[0109] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0110] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0111] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0113] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0116] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0117] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for refreshing a local cache, characterized in that: include: Preloading commonly used data into a local cache, wherein the commonly used data is obtained by analyzing historical log data; Controlling the amount of data in the local cache to be less than or equal to a capacity threshold of the local cache through a first strategy, and dynamically adjusting the lifetime of each data item in the local cache through a second strategy; Collect the data change records of the local cache, and asynchronously synchronize the data change records to the server, and refresh the local cache.

2. The method according to claim 1, characterized in that The controlling the amount of data in the local cache to be less than or equal to a capacity threshold of the local cache by using the first strategy includes: Generate a hash table and a bidirectional linked list according to each of the data items in the local cache; When receiving a data write request, obtaining a current amount of data in the local cache; If the current data volume is equal to the capacity threshold of the local cache, then removing the longest unused data item from the tail of the bidirectional linked list, and deleting the key-value pair corresponding to the longest unused data item in the hash table; When the current data amount is less than the capacity threshold of the local cache, a new data item corresponding to the data write request is added to the hash table and the head of the bidirectional linked list.

3. The method according to claim 1, characterized in that The dynamically adjusting the survival time of each data item in the local cache by using the second strategy includes: Monitoring the access frequency and cache hit rate of each data item in the local cache; The initial survival time of the data item is dynamically adjusted according to the access frequency and the cache hit rate, wherein the initial survival time is an initial value set when the data item is added to the cache.

4. The method according to claim 1, characterized in that: After preloading the commonly used data into the local cache, the method further includes: Monitoring the validity of each data item in the local cache; If a first data item is detected, new data is preloaded, wherein the first data item is the data item that is about to expire.

5. The method according to claim 1, characterized in that Before preloading the commonly used data into the local cache, the method further includes obtaining the commonly used data in the following manner: Calculating the access frequency of each data item in the historical log data, and performing cluster analysis on the historical log data to obtain multiple clusters; identifying access patterns of data items in each of the clusters; The common data is extracted from the historical log data according to the access frequency and the access pattern.

6. The method according to claim 1, characterized in that After dynamically adjusting the survival time of each of the data items in the local cache by using the second strategy, the method further includes: If a cache request from the target object is received, the cache request is parsed to determine a second data item to be searched; searching the local cache for the second data item; If the second data item exists in the local cache and the lifetime of the second data item has not expired, it is determined as a cache hit; if the second data item does not exist in the local cache, or the second data item exists in the local cache and the lifetime of the second data item has expired, it is determined as a cache miss; In case of a cache hit, reading the second data item, in case of a cache miss, obtaining the second data item from a data source, and adding the second data item to the local cache according to the first strategy; The second data item is sent to the target object.

7. The method according to claim 1, characterized in that The step of asynchronously synchronizing the data change record to the server and refreshing the local cache includes: When it is detected that the network connection state is stable, extracting the data change record in the task queue, wherein the task queue includes at least one of the collected data change records; generating a data request corresponding to the data change record, and sending the data request to the server; After receiving the response message from the server, the local cache is refreshed, and the processed data change records in the task queue are cleared.

8. A local cache refresh device, characterized in that: include: A loading module, used to preload commonly used data into a local cache, wherein the commonly used data is obtained by analyzing historical log data; A control module, configured to control the amount of data in the local cache to be less than or equal to a capacity threshold of the local cache through a first strategy, and to dynamically adjust the lifetime of each data item in the local cache through a second strategy; The synchronization module is used to collect the data change records of the local cache, and asynchronously synchronize the data change records to the server, and refresh the local cache.

9. An electronic device, comprising a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate through the communication bus and the communication interface, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method according to any one of claims 1 to 7.

Images