Cache management method and device based on tense perception and electronic equipment

Abstract

The invention relates to a cache management method and device based on tense perception and electronic equipment. The method comprises the following steps: acquiring log access data of a target cache item, and performing feature processing on the log access data to generate a time sequence feature and a user resource map feature; inputting the time sequence characteristics into a time sequence prediction model, outputting a first access probability, inputting the user resource map characteristics into an association prediction model, and outputting a second access probability; fusing the first access probability and the second access probability to obtain a comprehensive access probability of the target cache item; and inputting the comprehensive access probability and the state information of the cache system into a reinforcement learning model to enable the reinforcement learning model to output a target survival time value of the target cache item, and inputting the target survival time value into the cache system to dynamically set the survival time of the target cache item. The technical problems that cache management depends on manual rules, the maintenance cost is high, and self-adaption to dynamic changes of services and user behaviors is difficult are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a cache management method and device based on time perception and an electronic device. BACKGROUND

[0002] The life cycle management of the current mainstream cache system mainly depends on fixed Time To Live (TTL), access frequency or time eviction algorithm and simple hybrid strategy. The fixed TTL allocates the same survival time to each cache item, which is automatically deleted when expired. Although the implementation is simple, it cannot distinguish the access popularity of different data, for example, it is obviously unreasonable to use the same TTL for popular news and cold documents. The eviction algorithm based on access frequency or time, such as LRU (Least Recently Used) and LFU (Least Frequently Used), evicts the data with the lowest value according to the historical access record when the cache space is insufficient. However, it only relies on historical information and lacks the ability to predict future access trends, which can easily cause cache pollution or cache jitter. The simple hybrid strategy usually combines different TTL or rules with eviction algorithms. Although it optimizes cache management to some extent, it still relies on artificial rules, has high maintenance cost and is difficult to adapt to the dynamic changes of business and user behavior. SUMMARY

[0003] The present application provides a cache management method and device based on time perception and an electronic device to solve the technical problem that cache management relies on artificial rules, has high maintenance cost and is difficult to adapt to the dynamic changes of business and user behavior.

[0004] In a first aspect, the present application provides a cache management method based on time perception, comprising: obtaining log access data of a target cache item from a cache system, and performing feature processing on the log access data to generate time sequence features and user resource graph features; inputting the time sequence features into a time sequence prediction model to make the time sequence prediction model output a first access probability based on the historical access rule of the target cache item, and inputting the user resource graph features into an association prediction model to make the association prediction model output a second access probability based on the access relationship of the target cache item; fusing the first access probability and the second access probability to obtain a comprehensive access probability of the target cache item; inputting the comprehensive access probability and state information of the cache system into a reinforcement learning model to make the reinforcement learning model output a target survival time value of the target cache item, and inputting the target survival time value into the cache system to dynamically set the survival time of the target cache item.

[0005] In a second aspect, the application provides a cache management device based on time perception, comprising: an acquisition module configured to acquire log access data of a target cache item from a cache system, and perform feature processing on the log access data to generate time sequence features and user resource graph features; a first prediction module configured to input the time sequence features into a time sequence prediction model, so that the time sequence prediction model outputs a first access probability based on a historical access rule of the target cache item, and input the user resource graph features into an association prediction model, so that the association prediction model outputs a second access probability based on an access relationship of the target cache item; a fusion module configured to fuse the first access probability and the second access probability to obtain a comprehensive access probability of the target cache item; and a second prediction module configured to input the comprehensive access probability and state information of the cache system into a reinforcement learning model, so that the reinforcement learning model outputs a target survival time value of the target cache item, and input the target survival time value into the cache system to dynamically set a survival time of the target cache item.

[0006] As an optional example, the acquisition module comprises: an acquisition unit configured to acquire the log access data from the cache system, wherein the log access data comprises a cache key, a user identifier, an access timestamp, an operation type, and whether a hit occurs; a first processing unit configured to perform time sequence processing on the log access data based on the access timestamp to obtain time sequence features representing an access frequency of the target cache item; and a first construction unit configured to construct user resource graph features representing user access preferences based on an interaction relationship between the cache key and the user identifier, wherein an edge of the user resource graph features represents a user access behavior, and a weight of the edge represents an access frequency.

[0007] As an optional example, the first prediction module comprises: a first prediction unit configured to input the time sequence features into the time sequence prediction model, so that the time sequence prediction model learns periodicity, trendiness, and time dependence in the time sequence features, and predicts an access probability in a future preset time window to obtain the first access probability representing that the target cache item is accessed in the future preset time window.

[0008] As an optional example, the first prediction module comprises: a second prediction unit configured to input the user resource graph features into the association prediction model, so that the association prediction model learns embedded representations of a user node and a cache item node of the target cache item through a message passing mechanism, and predicts possible user access behaviors in a future preset time window according to the embedded representations to obtain the second access probability representing an association relationship.

[0009] As an optional example, the fusion module comprises: a second processing unit configured to: weight and average the first access probability and the second access probability according to a preset weight to obtain the comprehensive access probability; or splice the first access probability and the second access probability to form a feature vector, and input the feature vector into the fusion model, so that the fusion model performs nonlinear mapping on the feature vector to generate the comprehensive access probability.

[0010] As an optional example, the second prediction module comprises: a second construction unit configured to: construct the comprehensive access probability and state information of the cache system into a multi-dimensional state vector, wherein the state information comprises a current cache occupancy rate, an overall hit rate, and a feature of the target cache item; a calculation unit configured to: input the multi-dimensional state vector into the deep reinforcement learning model, so that the deep reinforcement learning model calculates a value or a policy probability of each candidate action in a candidate action set through a policy network or a value function, wherein each corresponding candidate action corresponds to a time-to-live value; and a determination unit configured to: determine a decision action from the candidate action set according to the value or the policy probability, to obtain the target time-to-live value.

[0011] As an optional example, the device further comprises: a monitoring module configured to: monitor access information of the target cache item in the cache system after inputting the target time-to-live value into the cache system to dynamically set the time-to-live of the target cache item, wherein the access information comprises an access result of the target cache item before the target time-to-live expires, and memory occupancy information of the target cache item; a quantization module configured to: quantize the access result and the memory occupancy information into a reward signal; and a feedback module configured to: feed back the reward signal to the reinforcement learning model to update a policy network or a value function of the reinforcement learning model.

[0012] In a third aspect, the present application provides a storage medium, wherein the storage medium stores a computer program, and the computer program is run by a processor to execute the cache management method based on time-awareness.

[0013] In a fourth aspect, the present application further provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the cache management method based on time-awareness by using the computer program.

[0014] Compared with the prior art, the above technical solutions provided by the embodiments of the present application have the following advantages:

[0015] The application obtains log access data of a target cache item from a cache system, and performs feature processing on the log access data to generate time sequence features and user resource graph features; the time sequence features are input into a time sequence prediction model to enable the time sequence prediction model to output a first access probability based on historical access rules of the target cache item, and the user resource graph features are input into a correlation prediction model to enable the correlation prediction model to output a second access probability based on an access relationship of the target cache item; the first access probability and the second access probability are fused to obtain a comprehensive access probability of the target cache item; the comprehensive access probability and state information of the cache system are input into a reinforcement learning model to enable the reinforcement learning model to output a target survival time value of the target cache item, and the target survival time value is input into the cache system to dynamically set the survival time of the target cache item. In the method, log access data of a target cache item is obtained from a cache system and feature processing is performed to generate time sequence features and user resource graph features. Secondly, time sequence features are input into a time sequence prediction model to generate a first access probability based on historical access rules, graph features are input into a correlation prediction model to generate a second access probability based on an access relationship, and the two are fused to obtain a comprehensive access probability of the target cache item. Finally, the comprehensive access probability and the state information of the cache system are input into a reinforcement learning model to calculate an optimal decision action corresponding to the target survival time value of the target cache item, and the target survival time value is dynamically set to the cache system, thereby realizing intelligent management of the survival time of the cache item, accurately predicting cache access trends, optimizing cache hit rate and memory utilization, reducing cache pollution and jitter, and improving the overall performance of the cache system. The technical problems of cache management relying on manual rules, high maintenance cost, and difficulty in adapting to dynamic changes in business and user behavior are solved. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, those skilled in the art can obtain other drawings from these drawings without any creative effort.

[0018] One or more embodiments are exemplarily illustrated by pictures in the drawings corresponding thereto, and these exemplary illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, unless otherwise specified. The drawings do not constitute a proportional limitation.

[0019] Figure 1 is a flow chart of an optional time-aware based cache management method according to an embodiment of the present application;

[0020] Figure 2 is an overall architecture correlation diagram of an optional time-aware based cache management method according to an embodiment of the present application;

[0021] Figure 3 is a data collection and feature engineering flow chart of an optional time-aware based cache management method according to an embodiment of the present application;

[0022] Figure 4 is a double-layer model prediction flow chart of an optional time-aware based cache management method according to an embodiment of the present application;

[0023] Figure 5 is a reinforcement learning decision and feedback loop diagram of an optional time-aware based cache management method according to an embodiment of the present application;

[0024] Figure 6 is a structural schematic diagram of an optional time-aware based cache management device according to an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0027] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For the purpose of simplicity, the description of a particular example will not necessarily be repeated in the description of each example. Of course, they are merely examples and are not intended to limit the present application. In addition, reference numerals and / or letters can be repeated in different examples. Such repetition is for the purpose of simplicity and clarity, and does not indicate a relationship between the various embodiments and / or arrangements being discussed.

[0028] According to a first aspect of the embodiments of the present application, a time-aware based cache management method is provided. Optionally, as shown in Figure 1 the above method comprises:

[0029] S102, obtain the log access data of the target cache item from the caching system, and perform feature processing on the log access data to generate time-series features and user resource graph features;

[0030] S104, input the time series features into the time series prediction model so that the time series prediction model outputs the first access probability based on the historical access pattern of the target cache item, and input the user resource graph features into the association prediction model so that the association prediction model outputs the second access probability based on the access relationship of the target cache item;

[0031] S106, combine the first access probability and the second access probability to obtain the comprehensive access probability of the target cache item;

[0032] S108, the comprehensive access probability and the state information of the caching system are input into the reinforcement learning model so that the reinforcement learning model outputs the target lifetime value of the target cache item, and the target lifetime value is input into the caching system to dynamically set the lifetime of the target cache item.

[0033] Optionally, in this embodiment, as Figure 2 The overall architecture diagram shown demonstrates how intelligent decision-making, based on the perception of historical access behavior of cached items, prediction of future access probabilities, and the combined effect of cache system status, enables dynamic lifecycle management of cached items, thereby optimizing cache hit rate and memory usage efficiency. This primarily involves three steps: data acquisition and feature engineering, two-layer model intelligent prediction, and reinforcement learning decision-making.

[0034] Step 1: Data Acquisition and Feature Engineering. First, log access data for the target cached item is obtained from the caching system. This log access data includes key information such as the cache key, user identifier, access timestamp, operation type (read / write), and whether the cache item was hit. Then, the collected log access data undergoes feature processing to generate structured feature information for model input. Specifically, two types of key features are generated: first, time-series features, which are generated by statistically analyzing the number of accesses to each cached item within different time windows to form time-series data reflecting the access patterns and historical trends of the cached items; second, user-resource graph features, which construct a dynamic "user-resource" heterogeneous graph based on the user identifier and the access relationship between cached items. In this graph, nodes represent users and cached items, edges represent user access behavior to cached items, and edge weights reflect access frequency or the relevance of recent accesses. This step transforms the raw unstructured log data into time-series and graph features usable by the predictive model, providing foundational data for the next prediction step.

[0035] Step two: intelligent prediction of double-layer model. The future access probability of the target cache item is predicted in multiple dimensions. First, the generated time series features are input into a time series prediction model (such as LSTM) to learn the historical access pattern of the cache item, and the first access probability based on time series is output, reflecting the trend of the cache item being accessed in future time windows. Second, the user resource graph features are input into an association prediction model (such as graph neural network GNN), which captures the deep association between user preferences and cache item attributes through node embedding and message passing mechanism, and outputs the second access probability based on access relationship, which is used to evaluate the potential access possibility of the target cache item in the user group. Then, the two types of access probabilities are fused, and the comprehensive access probability of the target cache item can be obtained by weighted average or feature vector splicing, so as to consider the influence of time pattern and user association in two dimensions, and improve the accuracy of access prediction.

[0036] Step three: reinforcement learning decision. The comprehensive access probability of the target cache item and the state information of the cache system (such as current cache occupancy rate, overall hit rate and cache item characteristics) are constructed into a multi-dimensional state vector and input into a deep reinforcement learning model (such as DQN or PPO). The reinforcement learning model calculates the value of each action in the candidate action set according to the policy network or the value function, and the candidate action corresponds to different TTL actions, each TTL action represents the specific survival time (such as 5 minutes, 30 minutes, 2 hours or 1 day) of the cache item. The optimal action is selected according to the policy network or the value function, and the decision action is output, and the TTL value corresponding to the decision action is the target survival time value of the target cache item. Then, the target survival time value is applied to the cache system to dynamically set the survival time of the target cache item, realizing intelligent cache management. Through continuous access monitoring and reward feedback, the reinforcement learning model can adaptively adjust the strategy, so that the TTL decision in similar future states is more accurate.

[0037] Optionally, in the embodiment, the intelligent and dynamic management of the cache item life cycle is realized. First, through the construction of time series features and graph features, the historical access behavior of the cache item and the user association relationship can be comprehensively perceived, providing rich data support for prediction. Second, the double-layer prediction module comprehensively evaluates the future access probability of the cache item from the time pattern and user relationship dimensions, significantly improving the prediction accuracy. Third, the reinforcement learning decision module autonomously selects the optimal TTL according to the prediction probability and system state, realizing the balance optimization of cache hit rate and memory occupancy. Finally, the method can dynamically adjust the survival time of the cache item, adapt to the changes of business and user behavior, reduce cache pollution and jitter phenomenon, improve the overall performance and resource utilization efficiency of the cache system, and reduce the burden of manual management.

[0038] As an optional example, log access data of the target cache item is obtained from the cache system, and the log access data is feature processed to generate time sequence features and user resource graph features, including:

[0039] The log access data is obtained from the cache system, wherein the log access data includes a cache key, a user identifier, an access timestamp, an operation type, and whether a hit;

[0040] Based on the access timestamp, the log access data is time-sequenced to obtain time sequence features representing the access frequency of the target cache item;

[0041] Based on the interaction relationship between the cache key and the user identifier, a user resource graph feature representing user access preference is constructed, wherein the edge of the user resource graph feature represents the access behavior of the user, and the weight of the edge represents the access frequency.

[0042] Optionally, in the present embodiment, as shown in the data collection and feature engineering flowchart, Figure 3 the log access data of the target cache item is first obtained from the cache system, and the log access data includes cache key (Cache Key), user identifier (User ID), access timestamp (Timestamp), operation type (read / write), and hit (Hit / Miss) and other key information. Subsequently, the collected log data is feature processed to generate two types of structured feature information to support the input of subsequent prediction and decision modules.

[0043] First, the log access data is time-sequenced based on the timestamp, and the number of accesses or the access frequency of each cache item in different time windows is counted to form time sequence features. The time sequence features can reflect the periodicity, trend and access hotspots of the cache item in historical access, providing a time dimension behavior reference for the prediction model.

[0044] Secondly, based on the interaction relationship between the cache key and the user identifier, a user resource graph feature is constructed. The graph is based on user nodes and cache item nodes, and the edge in the graph represents the user's access behavior to the cache item, and the weight of the edge is used to quantify the access frequency or recent access preference. Through the graph feature, the access mode of the user group and the association characteristics of the cache item among different users can be captured, providing data support for association prediction.

[0045] Through the above steps, the original unstructured access log can be converted into structured time sequence features and graph features that can be used by the model, realizing comprehensive perception of the historical access behavior of the cache item and the user access preference.

[0046] Optionally, in the embodiment, by constructing the time sequence feature and the user resource graph feature, the history access rule and the user access preference of the cache item can be comprehensively reflected, the accuracy of the prediction model for the future access behavior is improved, the deep perception of the cache access behavior is realized, the reliable basis for dynamically adjusting the cache item survival time is provided, the cache hit rate is optimized, the cache pollution and jitter are reduced, the overall performance of the cache system is improved, and the artificial management burden is reduced.

[0047] As an optional example, inputting the time sequence feature into the time sequence prediction model to make the time sequence prediction model output the first access probability based on the history access rule of the target cache item includes:

[0048] Inputting the time sequence feature into the time sequence prediction model to make the time sequence prediction model learn the periodicity, the trend and the time dependence in the time sequence feature, and predict the access probability in the future preset time window to obtain the first access probability representing that the target cache item is accessed in the future preset time window.

[0049] Optionally, in the embodiment, as shown in the double-layer model prediction flowchart, Figure 4 the time sequence feature of the target cache item is input into the time sequence prediction model to obtain the access probability based on the history access rule. First, the time sequence feature generated for the target cache item is processed, the feature reflects the access times or the access frequency of the cache item in different time windows, and can depict the periodicity, the trend and the time dependence of the access behavior. Then, the time sequence feature is input into the time sequence prediction model, such as the long short-term memory network (LSTM), so that the model can learn and capture the regularity information in the history access sequence, including the daily cycle, the weekly cycle and other long-term trends. In the model learning process, the time sequence prediction model encodes the access situation at each time point in the sequence, establishes the mapping relationship between the history access behavior and the future access probability, and predicts the access situation in the future preset time window in combination with the sequence context information. The prediction result is output in the form of the first access probability, representing the possibility that the target cache item is accessed in the future time window. The probability not only reflects the history access trend of the cache item itself, but also provides a quantitative basis for subsequent cache management, thereby supporting the dynamic TTL decision.

[0050] Optionally, in the embodiment, by inputting the time sequence feature into the time sequence prediction model and generating the first access probability, the history access rule of the cache item can be fully utilized for future access prediction, and the prediction accuracy and response speed are improved.

[0051] As an optional example, inputting the user resource graph feature into the association prediction model to make the association prediction model output the second access probability based on the access relationship of the target cache item includes:

[0052] The user-resource graph feature is input into the association prediction model, so that the association prediction model learns the embedding representation of the user node and the cache item node of the target cache item through a message passing mechanism, and predicts the user access behavior that may occur in a future preset time window according to the embedding representation to obtain a second access probability representing the association.

[0053] Optionally, in the embodiment, as shown in the double-layer model prediction flowchart, Figure 4 the user-resource graph feature is processed first, the feature is composed of user nodes and cache item nodes, and the edges represent user access to cache item behaviors and the edge weights represent access frequencies or recent access preferences. The graph can reflect the interaction between users and cache items and the group access mode, and provides structured information for association prediction. Then, the graph feature is input into the association prediction model, such as a graph neural network (GNN). The association prediction model learns the embedding representation of the user node and the cache item node through a message passing mechanism, thereby capturing the complex association between user access behaviors and the deep features of cache item attributes. During the model training process, the node embedding can reflect user preferences, access tendencies, and the relevance of cache items in different user groups, providing support for predicting future access behaviors. Based on the learned node embedding representation, the association prediction model predicts the user access behavior that may occur in a future preset time window to generate a second access probability. The probability is used to quantify the potential possibility of the target cache item being accessed in the user group, supplementing the association information that cannot be covered by simple time series prediction, thereby improving the comprehensiveness and accuracy of access prediction.

[0054] Optionally, in the embodiment, by inputting the user-resource graph feature into the association prediction model and outputting the second access probability, the association characteristics of user group access behaviors and cache items can be effectively captured, and the limitations of time series prediction based only on historical access sequences can be compensated.

[0055] As an optional example, fusing the first access probability and the second access probability to obtain a comprehensive access probability of the target cache item includes:

[0056] The first access probability and the second access probability are weighted and averaged according to a preset weight to obtain the comprehensive access probability; or

[0057] The first access probability and the second access probability are spliced to form a feature vector, and the feature vector is input into a fusion model, so that the fusion model performs nonlinear mapping on the feature vector to generate a comprehensive access probability.

[0058] Optionally, in the embodiment, as shown in the double-layer model prediction flowchart, Figure 4The shown double-layer model prediction flowchart fuses the first access probability obtained by the time-series prediction model and the second access probability obtained by the association prediction model to generate a comprehensive access probability of the target cache item. To fuse the two kinds of prediction information, two processing manners are provided. One manner is to perform weighted average on the first access probability and the second access probability according to a preset weight to obtain the comprehensive access probability, wherein the weight can be adjusted according to business targets or prediction accuracy to realize flexible balancing of different prediction sources. Another manner is to splice the first access probability and the second access probability to form a feature vector, and input the feature vector into a fusion model such as a multi-layer perception (MLP), so that the fusion model learns the complex relationship between the two kinds of probabilities through nonlinear mapping to generate a more accurate comprehensive access probability. The comprehensive access probability considers both the historical access regularity of the cache item itself and the access preference of the user group, and realizes comprehensive quantification of the future access possibility of the cache item.

[0059] Optionally, in the embodiment, by fusing the first access probability and the second access probability, the access information in the time dimension and the association dimension can be fully utilized to improve the accuracy and reliability of access prediction. The limitations of a single prediction model are effectively overcome, and multi-angle quantitative evaluation of the future access behavior of the cache item is realized.

[0060] As an optional example, inputting the comprehensive access probability and the state information of the cache system into a reinforcement learning model to make the reinforcement learning model output a target survival time value of the target cache item includes:

[0061] Constructing the comprehensive access probability and the state information of the cache system into a multi-dimensional state vector, wherein the state information includes a current cache occupancy rate, an overall hit rate, and characteristics of the target cache item;

[0062] Inputting the multi-dimensional state vector into a deep reinforcement learning model to make the deep reinforcement learning model calculate a value or a policy probability of each candidate action in a candidate action set through a policy network or a value function, wherein each corresponding candidate action corresponds to a survival time value;

[0063] According to the value or the policy probability, determining a decision action from the candidate action set to obtain the target survival time value.

[0064] Optionally, in the embodiment, as Figure 5The illustrated reinforcement learning decision and feedback loop diagram inputs the comprehensive access probability of the target cache item and the state information of the cache system into a reinforcement learning model to output a target time-to-live value of the target cache item. First, the comprehensive access probability and the state information of the cache system are constructed into a multi-dimensional state vector, and the state information includes the current cache occupancy rate, the overall hit rate, and the characteristics of the target cache item itself, such as the cache key, the access frequency, and the historical hit situation. These information can comprehensively reflect the real-time load of the cache system and the access characteristics of the cache item, and provide sufficient environmental perception for intelligent decision-making.

[0065] Subsequently, the constructed multi-dimensional state vector is input into a deep reinforcement learning model, such as a deep Q network (DQN) or a policy gradient method (such as PPO). The reinforcement learning model calculates the value or policy probability of each candidate action in the candidate action set through a policy network or a value function, where each candidate action corresponds to a specific time-to-live value (TTL). In the calculation process, the model comprehensively considers the cache hit benefit and the memory occupancy cost, and selects the optimal cache strategy by evaluating the candidate actions.

[0066] Finally, according to the value or policy probability of the candidate action, a decision action is determined from the candidate action set, thereby generating a target time-to-live value of the target cache item. The target time-to-live value can be directly applied to the cache system to realize dynamic expiration management of the cache item, and ensure efficient utilization of cache resources and optimization of access performance.

[0067] Optionally, in this embodiment, by inputting the comprehensive access probability and the cache system state information into the deep reinforcement learning model, intelligent TTL decision based on multi-dimensional environmental perception can be realized. The TTL of the cache item can be dynamically adjusted to balance the cache hit rate and the memory occupancy, reduce cache pollution and hotspot jitter, and improve the overall performance and resource utilization efficiency of the cache system.

[0068] As an optional example, after inputting the target time-to-live value into the cache system to dynamically set the TTL of the target cache item, the above method further includes:

[0069] Monitoring the access information of the target cache item in the cache system, wherein the access information includes the access result of the target cache item before the target time-to-live expires, and the memory occupancy information of the target cache item;

[0070] Quantifying the access result and the memory occupancy information into a reward signal;

[0071] Feeding back the reward signal to the reinforcement learning model to update the policy network or the value function of the reinforcement learning model.

[0072] Optionally, in this embodiment, as Figure 5The shown reinforcement learning decision and feedback loop figure inputs the target cache item target survival time value into the cache system to realize dynamic TTL management and continuously optimize the cache strategy through the feedback loop. First, the access information of the target cache item is monitored in the cache system in real time. The access information includes the access result of the target cache item before the target survival time expires, for example, whether it is successfully hit, and the memory occupation of the target cache item. These information can comprehensively reflect the actual value of the cache item and its occupation of system resources.

[0073] Subsequently, the monitored access result and memory occupation information are quantified as reward signals. The reward signal design balances the cache hit benefit and memory occupation cost. For example, when the cache item is accessed and hit within the effective TTL, a positive reward is given. When the cache item occupies memory but is not accessed, a negative reward or a reduced reward value is given. The reward signal is used as the feedback input of the reinforcement learning model to guide the strategy optimization.

[0074] Finally, the reward signal is fed back to the deep reinforcement learning model. The reinforcement learning model updates its policy network or value function according to the reward, thereby optimizing the TTL decision of the subsequent cache item. Through this closed-loop mechanism, the reinforcement learning model can continuously learn and adapt to the dynamic changes of the cache access mode, and realize the self-optimization of the cache strategy.

[0075] Optionally, in the present embodiment, by applying the target survival time value to the cache system and introducing the feedback loop based on the access result and memory occupation, adaptive optimization of cache management can be realized. While ensuring high cache hit rate, it reduces memory waste and cache jitter phenomenon, making the cache system more intelligent and efficient. At the same time, the reinforcement learning model can continuously iterate the policy network through feedback, realize the dynamic adjustment of the TTL of the cache item, thereby improve the overall performance, resource utilization efficiency and system stability of the cache system, and significantly reduce the cost of manual intervention.

[0076] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action order described, because according to the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0077] According to another aspect of the embodiments of the present application, a cache management device based on time perception is also provided, as shown in Figure 6 which comprises:

[0078] The acquisition module 602 is configured to acquire log access data of a target cache item from a cache system, and perform feature processing on the log access data to generate time sequence features and user resource graph features.

[0079] The first prediction module 604 is configured to input the time sequence features into a time sequence prediction model, so that the time sequence prediction model outputs a first access probability based on a historical access rule of the target cache item, and input the user resource graph features into an association prediction model, so that the association prediction model outputs a second access probability based on an access relationship of the target cache item.

[0080] The fusion module 606 is configured to fuse the first access probability and the second access probability to obtain a comprehensive access probability of the target cache item.

[0081] The second prediction module 608 is configured to input the comprehensive access probability and state information of the cache system into a reinforcement learning model, so that the reinforcement learning model outputs a target survival time value of the target cache item, and input the target survival time value into the cache system to dynamically set a survival time of the target cache item.

[0082] It should be noted that the acquisition module 602 in this embodiment can be configured to perform step S102 in the embodiments of the present application, the first prediction module 604 in this embodiment can be configured to perform step S104 in the embodiments of the present application, the fusion module 606 in this embodiment can be configured to perform step S106 in the embodiments of the present application, and the second prediction module 608 in this embodiment can be configured to perform step S108 in the embodiments of the present application.

[0083] As an optional example, the acquisition module includes:

[0084] The acquisition unit is configured to acquire log access data from the cache system, wherein the log access data includes a cache key, a user identifier, an access timestamp, an operation type, and whether a hit.

[0085] The first processing unit is configured to perform time sequence processing on the log access data based on the access timestamp to obtain time sequence features representing an access frequency of the target cache item.

[0086] The first construction unit is configured to construct user resource graph features representing user access preferences based on an interaction relationship between the cache key and the user identifier, wherein an edge of the user resource graph features represents an access behavior of a user, and a weight of the edge represents an access frequency.

[0087] As an optional example, the first prediction module includes:

[0088] The first prediction unit is configured to input the time sequence feature into a time sequence prediction model, so that the time sequence prediction model learns periodicity, trend and time dependence in the time sequence feature, and predicts the access probability in a preset future time window to obtain a first access probability representing access of the target cache item in the preset future time window.

[0089] As an optional example, the first prediction module includes:

[0090] The second prediction unit is configured to input the user resource graph feature into an association prediction model, so that the association prediction model learns embedded representations of the user node and the cache item node of the target cache item through a message passing mechanism, and predicts possible user access behaviors in a preset future time window according to the embedded representations to obtain a second access probability representing the association.

[0091] As an optional example, the fusion module includes:

[0092] The second processing unit is configured to weight average the first access probability and the second access probability according to a preset weight to obtain a comprehensive access probability; or

[0093] The first access probability and the second access probability are spliced to form a feature vector, and the feature vector is input into a fusion model, so that the fusion model performs nonlinear mapping on the feature vector to generate the comprehensive access probability.

[0094] As an optional example, the second prediction module includes:

[0095] The second construction unit is configured to construct the comprehensive access probability and state information of the cache system into a multi-dimensional state vector, wherein the state information includes a current cache occupancy rate, an overall hit rate and a self feature of the target cache item.

[0096] The calculation unit is configured to input the multi-dimensional state vector into a deep reinforcement learning model, so that the deep reinforcement learning model calculates a value or a policy probability of each candidate action in a candidate action set through a policy network or a value function, wherein each corresponding candidate action corresponds to a time-to-live value.

[0097] The determination unit is configured to determine a decision action from the candidate action set according to the value or the policy probability, to obtain a target time-to-live value.

[0098] As an optional example, the above device further includes:

[0099] The monitoring module is configured to monitor access information of the target cache item in the cache system after inputting the target time-to-live value into the cache system to dynamically set the time-to-live of the target cache item, wherein the access information comprises an access result of the target cache item before the target time-to-live expires and memory occupation information of the target cache item.

[0100] The quantifying module is configured to quantify the access result and the memory occupation information into a reward signal.

[0101] The feedback module is configured to feed back the reward signal to the reinforcement learning model to update a policy network or a value function of the reinforcement learning model.

[0102] Other examples of the embodiment are described in the above examples, which are not repeated here.

[0103] Figure 7 is a schematic diagram of an optional electronic device according to an embodiment of the present application, as shown in Figure 7 The electronic device includes a processor 702, a communication interface 704, a memory 706 and a communication bus 708, wherein the processor 702, the communication interface 704 and the memory 706 complete communication with each other through the communication bus 708, wherein

[0104] The memory 706 is configured to store a computer program.

[0105] The processor 702 is configured to implement the following steps when executing the computer program stored in the memory 706:

[0106] Obtain log access data of the target cache item from the cache system, and perform feature processing on the log access data to generate time sequence features and user resource graph features;

[0107] Input the time sequence features into a time sequence prediction model to make the time sequence prediction model output a first access probability based on a historical access rule of the target cache item, and input the user resource graph features into an association prediction model to make the association prediction model output a second access probability based on an access relationship of the target cache item;

[0108] Fuse the first access probability and the second access probability to obtain a comprehensive access probability of the target cache item;

[0109] Input the comprehensive access probability and state information of the cache system into a reinforcement learning model to make the reinforcement learning model output a target time-to-live value of the target cache item, and input the target time-to-live value into the cache system to dynamically set the time-to-live of the target cache item.

[0110] Optionally, in the embodiment, the communication bus described above can be a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For the convenience of representation, Figure 7 Only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus. The communication interface is used for communication between the electronic device and other devices.

[0111] The memory can include a RAM, and can also include a non-volatile memory, for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0112] As an example, the memory 706 described above can include, but is not limited to, the acquisition module 602, the first prediction module 604, the fusion module 606, and the second prediction module 608 in the time-aware based cache management apparatus described above. In addition, other module units in the time-aware based cache management apparatus described above can also be included, but not limited to, which will not be described herein.

[0113] The processor described above can be a general-purpose processor, which can include, but is not limited to, a CPU (Central Processing Unit), an NP (Network Processor), and the like; and can also be a DSP (Digital Signal Processing), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0114] Optionally, the specific examples in the embodiment can refer to the examples described in the above-described embodiments, which will not be described herein.

[0115] Those of ordinary skill in the art can understand that, Figure 7The structure shown is only schematic, and the device implementing the above-mentioned time-aware based cache management method can be a terminal device, which can be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, or the like. Figure 7 The structure of the above-mentioned electronic device is not limited. For example, the electronic device can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure. Figure 7 Figure 7 The structure of the above-mentioned electronic device is not limited. For example, the electronic device can further include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a different configuration from that shown in the figure.

[0116] Those skilled in the art can understand that all or part of the steps of the various methods of the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

[0117] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which stores a computer program, and when the computer program is run by a processor, the steps of the above-mentioned time-aware based cache management method are executed.

[0118] Optionally, in the present embodiment, those skilled in the art can understand that all or part of the steps of the various methods of the above-mentioned embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer readable storage medium, which can include a flash disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

[0119] The serial numbers of the above-mentioned embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0120] The integrated units in the above-mentioned embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above-mentioned computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing one or more computer devices (which can be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0121] ​In the above-described embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0122] In several embodiments provided in the present application, it should be understood that the disclosed client can be implemented by other manners. Among them, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interfaces, indirect coupling or communication connection between units or modules, and can be in electrical or other forms.

[0123] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0124] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0125] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A time-aware cache management method, characterized in that, include: The system obtains log access data of the target cache item from the caching system and performs feature processing on the log access data to generate time-series features and user resource graph features. The time-series features are input into the time-series prediction model so that the time-series prediction model outputs a first access probability based on the historical access patterns of the target cache item, and the user resource graph features are input into the association prediction model so that the association prediction model outputs a second access probability based on the access relationships of the target cache item. By combining the first access probability and the second access probability, the comprehensive access probability of the target cache item is obtained; The comprehensive access probability and the state information of the caching system are input into the reinforcement learning model so that the reinforcement learning model outputs the target lifetime value of the target cache item, and the target lifetime value is input into the caching system to dynamically set the lifetime of the target cache item.

2. The method according to claim 1, characterized in that, The process involves retrieving log access data for the target cache item from the caching system, performing feature processing on the log access data, and generating time-series features and user resource graph features, including: The log access data is obtained from the caching system, wherein the log access data includes a cache key, user identifier, access timestamp, operation type, and whether the cache hit occurred. Based on the access timestamp, the log access data is processed by time serialization to obtain time-series characteristics representing the access frequency of the target cache item; Based on the interaction between the cache key and the user identifier, a user resource graph feature representing the user's access preferences is constructed, wherein the edges of the user resource graph feature represent the user's access behavior, and the weight of the edge represents the access frequency.

3. The method according to claim 1, characterized in that, The time-series features are input into the time-series prediction model so that the time-series prediction model outputs a first access probability based on the historical access patterns of the target cache item, including: The time-series features are input into the time-series prediction model so that the time-series prediction model can learn the periodicity, trend and time dependence in the time-series features and predict the access probability within a future preset time window to obtain a first access probability characterizing the target cache item being accessed in the future preset time window.

4. The method according to claim 1, characterized in that, The user resource graph features are input into the association prediction model so that the association prediction model outputs a second access probability based on the access relationship of the target cache item, including: The user resource graph features are input into the association prediction model, so that the association prediction model learns the embedding representations of the user nodes and cache item nodes of the target cache item through a message passing mechanism, and predicts the user access behavior that may occur within a future preset time window based on the embedding representations, thereby obtaining a second access probability representing the association relationship.

5. The method according to claim 1, characterized in that, The combined access probability of the target cache item, obtained by integrating the first access probability and the second access probability, includes: The first access probability and the second access probability are weighted and averaged according to a preset weight to obtain the comprehensive access probability; or The first access probability and the second access probability are concatenated to form a feature vector, and the feature vector is input into the fusion model so that the fusion model performs a nonlinear mapping on the feature vector to generate the comprehensive access probability.

6. The method according to any one of claims 1 to 5, characterized in that, The comprehensive access probability and the state information of the caching system are input into the reinforcement learning model so that the reinforcement learning model outputs the target lifetime value of the target cache item, including: The comprehensive access probability and the state information of the caching system are used to construct a multi-dimensional state vector, wherein the state information includes the current cache occupancy rate, the overall hit rate and the characteristics of the target cache item. The multidimensional state vector is input into the reinforcement learning model so that the reinforcement learning model calculates the value or policy probability of each candidate action in the candidate action set through a policy network or a value function, wherein each corresponding candidate action corresponds to a survival time value; Based on the value or the strategy probability, a decision action is determined from the set of candidate actions to obtain the target survival time value.

7. The method according to claim 6, characterized in that, After inputting the target lifetime value into the caching system to dynamically set the lifetime of the target cache item, the method further includes: The access information of the target cache item is monitored in the caching system, wherein the access information includes the access results of the target cache item before the target life time expires, and the memory usage information of the target cache item; The access results and memory usage information are quantified into a reward signal; The reward signal is fed back to the reinforcement learning model to update the policy network or value function of the reinforcement learning model.

8. A time-aware cache management device, characterized in that, include: The acquisition module is used to acquire log access data of the target cache item from the caching system, and perform feature processing on the log access data to generate time-series features and user resource graph features. The first prediction module is used to input the time-series features into the time-series prediction model so that the time-series prediction model outputs a first access probability based on the historical access patterns of the target cache item, and to input the user resource graph features into the association prediction model so that the association prediction model outputs a second access probability based on the access relationships of the target cache item. The fusion module is used to fuse the first access probability and the second access probability to obtain the comprehensive access probability of the target cache item; The second prediction module is used to input the comprehensive access probability and the state information of the caching system into the reinforcement learning model, so that the reinforcement learning model outputs the target survival time value of the target cache item, and inputs the target survival time value into the caching system to dynamically set the survival time of the target cache item.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to perform the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 7 through the computer program.

Images