Data caching method, device and equipment, network equipment and storage medium

Abstract

The invention discloses a data caching method, device and equipment, network equipment and a storage medium. The method comprises the following steps: generating spectrogram embedded data according to obtained system data of a distributed caching system; determining a spectrogram embedding vector for representing the content similarity and the position similarity among the nodes according to an attention mechanism through the spectrogram embedding data and the state information of the nodes in the system data; and determining a cache replacement strategy according to the spectrogram embedding vector and the access information stored by each node in the distributed cache system. According to the data caching method provided by the embodiment of the invention, the problem that the caching decision is not accurate enough because the influence of the network topology on the data access efficiency is ignored in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and specifically to a data caching method, apparatus, device, network device, and storage medium. Background Art

[0002] With the continuous growth of data volumes and the increasing complexity of processing requirements, efficient distributed cache management has become a key factor in improving system performance. Distributed cache not only significantly reduces data access latency but also effectively reduces network bandwidth pressure and the load on backend storage systems. However, in practical applications, distributed cache faces many challenges, such as maintaining cache consistency, load balancing, and dynamically changing data access patterns.

[0003] Currently, solutions to the distributed cache management problem include dynamic adaptive caching algorithms. These primarily focus on cache optimization within a single node or a local scope, neglecting the network structure characteristics of the entire distributed system. While they can adjust caching strategies based on real-time conditions, they struggle to strike a good balance between global optimization and local responsiveness. They also ignore the impact of network topology on data access efficiency, resulting in inaccurate caching decisions. Summary of the Invention

[0004] At least one embodiment of the present application provides a data caching method, apparatus, device, network device and storage medium for solving the problem in the prior art of distributed cache management that ignores the impact of network topology on data access efficiency, resulting in inaccurate cache decisions.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a data caching method, comprising:

[0007] Generate spectrum embedding data based on the obtained system data of the distributed cache system; the spectrum embedding data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system;

[0008] Determining, using the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for characterizing the content similarity and position similarity between the nodes according to an attention mechanism; the content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes;

[0009] A cache replacement strategy is determined based on the spectral graph embedding vector and the access information stored by each node in the distributed cache system; the cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

[0010] Furthermore, the spectrum embedding data is generated based on the obtained system data of the distributed cache system, including:

[0011] Determining a dynamic graph corresponding to the distributed cache system, wherein the dynamic graph is used to represent a relationship between communication links or data transmission paths between nodes that changes over time;

[0012] Constructing a Laplace matrix of the distributed cache system according to the dynamic graph;

[0013] Perform spectral decomposition on the Laplace matrix to generate the spectral graph embedding data.

[0014] Furthermore, the system data includes:

[0015] The network topology information of the distributed cache system, the real-time status information of each node, and the data access mode of the distributed cache system;

[0016] The network topology information is used to indicate the connection relationship between the nodes in the distributed cache system;

[0017] The real-time status information includes the storage capacity, computing resources, current cache content and load status of each node;

[0018] The data access pattern includes the frequency and order in which data is accessed in the distributed cache system.

[0019] Furthermore, determining a spectral embedding vector for characterizing content similarity and position similarity between nodes according to an attention mechanism using the spectral embedding data and the state information of the nodes in the system data includes:

[0020] Determine a state vector based on the spectrogram embedded data and the state information; the state vector includes: node access data, cache content, access history, and a correspondence between the spectrogram embedded data and time;

[0021] Determining a spectral graph attention function based on the state vector and the attention mechanism; the spectral graph attention function is related to the content similarity and position similarity between each node;

[0022] The spectrogram embedding vector is determined according to the spectrogram attention function.

[0023] Furthermore, a cache replacement strategy is determined based on the spectral embedding vector and the access information stored by each node in the distributed cache system, including:

[0024] Determining access status and importance of stored data items in the distributed cache system using Belady's algorithm based on the spectral graph embedding vector and the access information stored in each node;

[0025] A cache replacement strategy including eliminated data items is determined based on access conditions and importance of data items stored in the distributed cache system.

[0026] Furthermore, the method further comprises:

[0027] Determining a reward function for the cache replacement strategy using an inverse reinforcement learning algorithm;

[0028] Training the policy network of the cache replacement strategy according to the reward function to obtain a target cache strategy; the policy network is used to represent the corresponding relationship between the data to be cached and the node state;

[0029] According to the target cache strategy, a cache scheme for the data to be cached in each node is determined.

[0030] Furthermore, the reward function of the cache replacement strategy is determined by an inverse reinforcement learning algorithm, including:

[0031] According to the inverse reinforcement learning algorithm, maximizing the likelihood probability of the cache replacement strategy trajectory is determined as the reward function.

[0032] Furthermore, the policy network of the cache replacement strategy is trained according to the reward function to obtain a target cache strategy, including:

[0033] According to the reward function, the policy network is trained using a policy gradient method to determine network parameters of the policy network in the cache replacement strategy;

[0034] The target caching strategy is determined according to the network parameters.

[0035] In a second aspect, an embodiment of the present application provides a data caching device, comprising:

[0036] A generation module, configured to generate spectrogram embedded data based on the obtained system data of the distributed cache system; the spectrogram embedded data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system;

[0037] A first determination module is configured to determine, based on the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for representing the content similarity and position similarity between each node according to an attention mechanism; the content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes;

[0038] The second determination module is used to determine a cache replacement strategy based on the spectrum embedding vector and the access information stored by each node in the distributed cache system; the cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

[0039] In a third aspect, an embodiment of the present application provides a data cache device, including: a processor;

[0040] The processor is configured to generate spectrogram embedded data based on the acquired system data of the distributed cache system; the spectrogram embedded data is configured to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system;

[0041] Determining, using the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for characterizing the content similarity and position similarity between the nodes according to an attention mechanism; the content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes;

[0042] A cache replacement strategy is determined based on the spectral graph embedding vector and the access information stored by each node in the distributed cache system; the cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

[0043] In a fourth aspect, an embodiment of the present application provides a network device comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the data caching method as described in the first aspect.

[0044] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the program implements the steps of the data caching method described above.

[0045] In a sixth aspect, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the data caching method described above.

[0046] Compared with the prior art, the data caching method, apparatus, device, network device, and storage medium provided in the embodiments of the present application can generate spectral embedding data by acquiring system data of a distributed cache system, and determine the relationship between each node and the global characteristics of the network topology of the distributed cache system based on the generated spectral embedding data; the spectral embedding vector can be used to determine the similarity and positional relationship between computing nodes, thereby improving the accuracy of decision-making; and the optimal cache replacement strategy is determined by combining the spectral embedding vector with the access information stored by each node. This solves the problem in the prior art that the impact of network topology on data access efficiency is ignored, resulting in inaccurate cache decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0048] Figure 1 A schematic diagram of the steps of the data caching method according to an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the coordinates of each node spectrum embedded in the embodiment of the present application;

[0050] Figure 3 Schematic diagram of the implementation framework of the data caching method according to an embodiment of the present application;

[0051] Figure 4 This is a module diagram of a data cache device according to an embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of the structure of a data cache device according to an embodiment of the present application;

[0053] Figure 6 This is a schematic diagram of the structure of the network device according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0055] The terms "first", "second" etc. in the specification and claims of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprise" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or that are intrinsic to these processes, methods, products or equipment. "And / or" in the specification and claims represents at least one of the connected objects.

[0056] In today's big data era, distributed systems have become the primary architecture for processing massive amounts of data and complex computing tasks. With the continuous growth of data volumes and the increasing complexity of processing requirements, efficient distributed cache management has become a key factor in improving system performance. Distributed caches not only significantly reduce data access latency, but also effectively reduce network bandwidth pressure and the load on backend storage systems. However, in practical applications, distributed caches face many challenges, such as maintaining cache consistency, load balancing, and dynamically changing data access patterns. Designing an adaptive, efficient, and scalable distributed cache system, especially in large-scale, highly concurrent environments, has become a major concern for both academia and industry.

[0057] Currently, solutions to distributed cache management problems can be mainly divided into the following categories: static rule-based cache strategies, dynamic adaptive cache algorithms, cache optimization methods based on machine learning and deep learning, and multi-level cache collaborative management strategies. Although static rule-based methods are simple to implement, they are difficult to adapt to complex and changing data access patterns. Dynamic adaptive algorithms can adjust cache strategies according to real-time conditions, but it is difficult to achieve a good balance between global optimization and local response. Although machine learning / deep learning methods have greatly improved cache hit rates in many scenarios, they still face challenges in scalability and computational efficiency in large-scale distributed environments. Multi-level cache collaborative management strategies can comprehensively utilize the storage characteristics of different levels, but how to achieve efficient collaboration between levels remains an open problem. In addition, in a distributed environment, how to effectively use network topology information to optimize cache decisions is also an aspect that is rarely addressed in existing solutions.

[0058] Therefore, the existing solutions to the distributed cache management problem mainly have the following problems:

[0059] (1) Lack of full utilization of the network topology of distributed systems. Most existing technologies focus on cache optimization within a single node or a local area, ignoring the network structure characteristics of the entire distributed system. In actual large-scale distributed environments, network topology has a significant impact on data access efficiency. For example, some nodes may be in key positions in the network and undertake more data forwarding tasks, while some nodes may be located at the edge and are mainly responsible for local data processing. Ignoring these structural characteristics may lead to inaccurate cache decisions and inability to fully utilize network resources, thereby affecting the overall system performance.

[0060] (2) There are deficiencies in dynamic adaptive capabilities. Although some methods have introduced machine learning techniques to optimize caching strategies, they are often trained based on static or semi-static data access patterns. However, in actual distributed systems, data access patterns may change rapidly over time and be affected by multiple factors such as user behavior, business logic, and even network conditions. Existing methods lack the ability to learn in real time and quickly adapt to these changes, resulting in performance degradation in dynamic environments. In addition, most methods do not take into account real-time conditions such as system load and network bandwidth, and are unable to flexibly adjust caching strategies based on the dynamic characteristics of the system.

[0061] (3) Challenges in scalability and system overhead. Although some methods based on complex machine learning models perform well in small-scale tests, they may face problems such as high computational complexity and difficulty in model updating in large-scale distributed environments. This not only increases the system's computational and storage overhead, but may also affect the real-time performance of cache decisions. In addition, some methods require frequent global information exchange or centralized decision-making processes, which may lead to serious communication overhead and performance bottlenecks in distributed systems with a large number of nodes. How to ensure the scalability and low overhead of the method in a large-scale distributed environment while ensuring the cache optimization effect is an urgent problem to be solved.

[0062] To solve at least one of the above problems, embodiments of the present application provide a data caching method, apparatus, device, network device, and storage medium to improve the accuracy of caching decisions.

[0063] The following description provides examples and does not limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. The various examples may appropriately omit, substitute, or add various procedures or components. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0064] like Figure 1As shown, a data caching method provided in an embodiment of the present application includes the following steps:

[0065] Step 101: Generate spectrogram embedded data based on the acquired system data of the distributed cache system; the spectrogram embedded data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system;

[0066] Step 102: Determine, using the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for characterizing the content similarity and position similarity between the nodes according to an attention mechanism; the content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes;

[0067] Step 103: Determine a cache replacement strategy based on the spectral graph embedding vector and the access information stored by each node in the distributed cache system; the cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

[0068] The data caching method of the present embodiment generates spectral graph embedding data from acquired distributed cache system system data, and determines the relationships between nodes and the global characteristics of the distributed cache system's network topology based on the generated spectral graph embedding data. The spectral graph embedding vectors are used to determine the similarities and positional relationships between computing nodes, improving decision-making accuracy. The spectral graph embedding vectors are combined with the access information stored by each node to determine the optimal cache replacement strategy. This method solves the problem in the prior art of ignoring the impact of network topology on data access efficiency, resulting in inaccurate caching decisions.

[0069] Optionally, the obtained system data of the distributed cache system includes:

[0070] The network topology information of the distributed cache system, the real-time status information of each node, and the data access mode of the distributed cache system;

[0071] The network topology information is used to indicate the connection relationship between the nodes in the distributed cache system;

[0072] The real-time status information includes the storage capacity, computing resources, current cache content and load status of each node;

[0073] The data access pattern includes the frequency and order in which data is accessed in the distributed cache system.

[0074] In the embodiments of this application, the performance of the system and the accuracy of its decision-making are highly dependent on the comprehensiveness and accuracy of the acquired system data. The system data not only covers the static structure of the system, but also its dynamic operating state, providing sufficient information support for subsequent system spectrum embedding data and optimization.

[0075] Specifically, the network topology information is the core data that describes the connection relationship between nodes in the distributed system. Each node in the distributed system can be regarded as a vertex v in the graph G(V,E). i ∈V, and the connection relationship between nodes is represented by the edge e ij ∈E. If the node v i and v j There is a connection relationship between them (such as data transmission or communication link), then e ij =1, otherwise e ij =0.

[0076] This network topology information not only provides the foundational data for subsequent spectral graph embedding, but also helps reveal the physical and logical relationships between nodes and capture the network's global structural characteristics. By processing this topological information, the system can identify key nodes and bottleneck nodes in the network, as well as their importance to the overall system.

[0077] Specifically, the real-time status information of each node refers to the current operating status of each node in the distributed system, including the node's storage capacity, computing resources (such as CPU, memory usage), current cache content, and node load.

[0078] Optionally, use the state vector s i (t) to represent the node v i The state at time t:

[0079] S i (t)=[c i (t),u cpu (t),u mem (t),cache(t),load(t)]

[0080] Among them, c i (t) represents the remaining storage capacity of the node, u cpu (t) and u mem (t) represents the node's CPU and memory usage, cache(t) represents the current cached content, and load(t) represents the node's current load (such as the number or frequency of requests being processed). This state information provides an important basis for subsequent caching decisions, allowing the system to make reasonable resource allocation and cache adjustments based on the node's actual status.

[0081] Collecting real-time node status enables the system to dynamically adapt to runtime load changes. For example, when a node's load is too high, the system can reduce cache pressure on that node or migrate some cached data to a less-loaded node. This status information can also be used to predict future node behavior, enabling proactive adjustments to cache replacement policies.

[0082] Specifically, the data access pattern is a set of rules and characteristics of data access (including request, read or write) in a distributed system. The data access pattern generally includes access frequency, access sequence, hot data distribution, etc.

[0083] It should be noted that the hot data distribution is determined according to the access frequency, and data with an access frequency greater than a preset threshold is hot data.

[0084] Among them, an access matrix P(t) can be used to represent the set of rules and characteristics of data access. The specific form is as follows:

[0085]

[0086] Where m is the number of nodes in the distributed system, n is the number of data items, and P ij (t) represents the time period of node v i For data d j By analyzing the access matrix P(t), we can identify the hot data in the system. These data are likely to be frequently accessed in the future and thus given priority in cache decisions.

[0087] Optionally, generating spectrum embedding data based on the acquired system data of the distributed cache system includes:

[0088] Determining a dynamic graph corresponding to the distributed cache system, wherein the dynamic graph is used to represent a relationship between communication links or data transmission paths between nodes that changes over time;

[0089] Constructing a Laplace matrix of the distributed cache system according to the dynamic graph;

[0090] Perform spectral decomposition on the Laplace matrix to generate the spectral graph embedding data.

[0091] It's important to note that in a distributed cache system, each cache node can be viewed as a vertex V in a graph G, while the communication links or data transmission paths between nodes correspond to edges E in the graph. Therefore, a distributed cache system can be described as a graph G(V, E). Under the influence of time t, G(V, E, t) is a dynamic graph, whose node set V and edge set E may change over time. This dynamic graph effectively describes the dynamic changes in node connectivity in a distributed system.

[0092] In order to further analyze the structural characteristics of the graph G(V, E, t), it is necessary to introduce the Laplacian matrix of the graph to construct the Laplacian matrix of the distributed cache system;

[0093] The Laplace matrix is defined as follows:

[0094] L(t)=D(t)-A(t)

[0095] Where D(t) is the degree matrix of graph G at time t, and A(t) is the adjacency matrix of G at time t. The degree matrix D(t) is a diagonal matrix, where the elements D on the diagonal are ii (t) represents the number of edges connected to node i (i.e., the degree of node i). The adjacency matrix A(t) is a symmetric matrix, where A ij (t) = 1 means that there is an edge connection (data transmission or communication link) between node i and node j, A ij (t)=0 means there is no direct connection between the two nodes.

[0096] The Laplacian matrix plays a crucial role in graph theory, reflecting the global properties of a graph. For example, the eigenvalue spectrum of the Laplacian matrix (the set of its eigenvalues) contains key information such as the graph's connectivity and clustering coefficient. Therefore, analyzing the Laplacian matrix provides insight into the network structure of distributed cache systems.

[0097] After establishing the Laplace matrix L(t) of the distributed cache system, the spectral decomposition is used to perform eigenvalue decomposition on L(t). Specifically, spectral decomposition solves the following eigenvalue problem:

[0098] L(t)v i (t) = λ i (t)v i (t)

[0099] Among them, λ i (t) represents the i-th eigenvalue, v iDenotes the corresponding eigenvector. By performing spectral decomposition on the Laplace matrix L(t), we can obtain a set of eigenvalues [λ1(t),λ2(t),…,λ n (t)] and the corresponding eigenvectors [v1(t),v2(t),…,v n (t)].

[0100] In the process of constructing spectral graph embedding data in a distributed cache system, the core role of spectral decomposition is to generate spectral graph embedding of nodes. Spectral graph embedding uses the eigenvectors of the Laplacian matrix to create a low-dimensional coordinate representation Φ(v i ,t), the coordinate can reflect the position and importance of the node in the entire network.

[0101] Specifically, node v i The spectral graph embedding Φ(v i ,t) can be represented by a linear combination of the first k eigenvectors:

[0102] Φ(v i ,t)=[v1(t) i ,v2(t) i ,…,v k (t) i ]

[0103] Among them, v j (t) i Represents feature v j The ith component of (t). Usually, the eigenvectors corresponding to the k smallest non-zero eigenvalues (set as needed, and k is less than the number of nodes in the distributed cache system) are selected for embedding to ensure that the main structural features of the graph can be captured.

[0104] To understand the process of spectral graph embedding more intuitively, let’s use a simple distributed cache system to explain it. The distributed cache system consists of five nodes A, B, C, D, and E. The connection relationship between them can be represented by the adjacency matrix A(t), and the corresponding degree matrix D(t) is:

[0105]

[0106] By definition, the Laplacian matrix L(t) can be obtained by subtracting the adjacency matrix A(t) from the degree matrix D(t):

[0107]

[0108] Next, perform eigenvalue decomposition on the Laplace matrix L(t) to obtain the eigenvalue λ i (t) and the eigenvector vi (t):

[0109]

[0110] Select the eigenvectors corresponding to the first two smallest non-zero eigenvalues λ2(t) and λ3(t) to obtain the embedded coordinates of each node spectrum graph: A[0.81 0.00], B[0.14 0.00], C[-0.26-0.71], D[-0.26 0.71], E[-0.44 0.00], as shown in Figure 2 As shown in Figure 2, spectral graph embedding provides a unique coordinate representation for each node in the distributed cache network. Through these coordinates, the relative position and importance of nodes can be quantitatively analyzed.

[0111] The data caching method of the embodiment of the present application can not only characterize the position representation of the node in the network through the spectrum embedding data, but also serve as an important basis for cache optimization decisions. In a distributed cache system, the relative position and connection relationship between different nodes have an important impact on the selection of cache strategy; for example, when deciding whether a data item needs to be cached at the current node, the position of the node in the network and the relative relationship with other nodes can be considered. If the spectrum embedding data shows that the current node is connected to multiple high-importance nodes, it can be preferred to cache the data item at this node to improve data accessibility and cache hit rate. By introducing spectrum embedding, the cache optimization algorithm can better understand and utilize the topological structure in the network, and then make more intelligent cache decisions. In addition, spectrum embedding can also be used to optimize data distribution and migration strategies; for example, in a distributed system, when a node is highly loaded or about to go offline, the system can migrate the data of the node to a node close to its position based on the spectrum embedding to reduce network communication overhead and ensure the stability and efficiency of the system.

[0112] Optionally, determining a spectral embedding vector for characterizing content similarity and position similarity between nodes according to an attention mechanism using the spectral embedding data and the state information of the nodes in the system data includes:

[0113] Determine a state vector based on the spectrogram embedded data and the state information; the state vector includes: node access data, cache content, access history, and a correspondence between the spectrogram embedded data and time;

[0114] Determining a spectral graph attention function based on the state vector and the attention mechanism; the spectral graph attention function is related to the content similarity and position similarity between each node;

[0115] The spectrogram embedding vector is determined according to the spectrogram attention function.

[0116] In the embodiment of the present application, the state information is a time-related state vector, for example, the state information is represented as the state vector s of each node at a given time t. i (t):

[0117] s i (t)=[a i (t),c i (t),h i (t),Φ(v i ,t)]

[0118] Among them, s i (t) consists of the following four key parts:

[0119] Current access dataa i (t): represents the data item that node i requests to access at time t (current);

[0120] Cache content c i (t): represents the data set in the cache of node i at time t;

[0121] Visit History i (t): represents the access record of node i over a period of time (time series or frequency vector); this information can capture the access pattern of the node and help predict future access demand;

[0122] Spectral embedding Φ(v i ,t): represents the coordinates of node i in the distributed cache network (provides the location and importance information of the node in the global network);

[0123] Node state vector representation s i (t) provides a comprehensive view for each node, enabling it to make caching decisions based on current demand, cache content, historical access patterns, and network topology.

[0124] At the same time, in order to enable nodes to effectively utilize the information of other nodes in the network, the spectral attention mechanism is introduced, which calculates the attention weight based on the topological relationship and state similarity between nodes.

[0125] In the embodiment of the present application, the spectrogram attention function is as follows:

[0126]

[0127] Among them, Q, K and V represent query, key and value matrices respectively, d kis the dimension of the key vector, α is an adjustable parameter used to balance content similarity and position similarity, Φ is the spectral graph embedding matrix, and its row vector is the spectral graph embedding vector Φ(v i ,t).

[0128] Among them, content similarity refers to the content similarity between the computing node's own state and other nodes' states, that is, the node and other nodes in s i (t) similarity;

[0129] Position similarity is used to measure the proximity of nodes to ensure that the network topology is considered when making decisions; for example, nodes with similar locations will be given higher weights when making decisions.

[0130] The data caching method of the embodiment of the present application introduces an attention mechanism so that when node i makes a caching decision, it can not only be based on its own state s i (t) can also effectively integrate the information of other nodes and refer more to the cache status of nodes that are close to it and have related content to determine whether to cache a data item.

[0131] Optionally, determining a cache replacement strategy based on the spectrum embedding vector and access information stored by each node in the distributed cache system includes:

[0132] Determining access status and importance of stored data items in the distributed cache system using Belady's algorithm based on the spectral graph embedding vector and the access information stored in each node;

[0133] A cache replacement strategy including eliminated data items is determined based on access conditions and importance of data items stored in the distributed cache system.

[0134] It should be noted that Belady's algorithm, also known as the optimal cache replacement algorithm, is a classic theoretical benchmark for cache replacement strategies. The core idea of this algorithm is to eliminate pages that will not be accessed for the longest period of time in the future to ensure the highest cache hit rate. However, the optimality of Belady's algorithm relies on complete knowledge of future access sequences.

[0135] In a single-node environment, Belady's algorithm can be expressed as: given the current cache state C(t) and the future access sequence {a t+1 ,a t+2 ,…}, at time t, if we want to convert data item a t If the cache is full, the latest data item in the cache that will be accessed in the future should be eliminated. out ,Right now:

[0136]

[0137] Among them, Index(a,{a t+1 ,a t+2 ,…}) represents the position index of the first appearance of data item $a$ in the future access sequence.

[0138] However, in a distributed cache system, which consists of multiple cache nodes, Belady's algorithm for a single node cannot be directly applied. Data access patterns, cache states, and network structure information vary significantly between nodes. In this case, caching decisions must not only consider the future access situation of local nodes but also the global state and data distribution of the entire network. Therefore, Belady's algorithm needs to be extended to meet the needs of a distributed environment.

[0139] In a distributed environment, assuming that the system consists of N nodes, the node set is represented by V = {v1, v2, ..., v N}. Each node v i The state at time t is represented by s i (t), the connection relationship between nodes is described by the adjacency matrix A(t). At this time, the goal of the distributed Belady's algorithm is to design a global strategy (the cache replacement strategy) π * (a t |S(t),G(t)), where S(t) represents the state set of all nodes and G(t) represents the topology of the entire network.

[0140] The innovation of the distributed Belady's algorithm proposed in this proposal is that it considers both the cache status of local nodes and the future access pattern of the entire network. In this extension, the distributed Belady's algorithm not only needs to predict the future access sequence of a single node, but also needs to estimate the "reuse pattern" of data globally through network topology information and interactions between nodes. For example, suppose that both node A and node B have cached data item X, but node A is not expected to access X again in the future, while node B will soon access X again. In this case, node A is more inclined to eliminate X because it expects node B to continue caching the data item and B can make it available to other nodes when needed. This global reuse consideration enables the distributed Belady's algorithm to better coordinate the cache resources of each node, thereby achieving optimal utilization of cache resources globally. Specifically, for a certain node v i , if its cache is full and needs to cache a new data item a t , then a data item a should be selected outThe selection of the data item is not only based on its future access situation at this node, but also on its reusability in the network. The calculation method is:

[0141]

[0142] Among them, G Reuse (a, S(t), G(t)) represents the reuse estimate of data item a in the global network, and λ is a tuning parameter that weighs local and global information.

[0143] G Reuse The calculation method of (a, S(t), G(t)) is:

[0144]

[0145] This formula evaluates the importance of a data item by considering both local and global reusability;

[0146] Among them, the local reusability is calculated first, that is, the data item a is calculated at the current node v i The future visit situation is recorded as R local (a,v i ,t). R local Is the data item a at node v i The position index in the future access sequence. Secondly, the calculation of global reusability refers to evaluating the future access requirements of data item a on other nodes. For each other node v j (j≠i), calculate the access index R of the node global (a, S(t), G(t)), and weighted processing combined with network topology information. The specific weight w ij Reflects the node v j For node v i The impact of the network distance d between nodes ij Inversely proportional, that is where ∈ is a constant that prevents the denominator from being zero.

[0147] Optionally, the method further includes:

[0148] Determining a reward function for the cache replacement strategy using an inverse reinforcement learning algorithm;

[0149] Training the policy network of the cache replacement strategy according to the reward function to obtain a target cache strategy; the policy network is used to represent the corresponding relationship between the data to be cached and the node state;

[0150] According to the target cache strategy, a cache scheme for the data to be cached in each node is determined.

[0151] Although distributed Belady's algorithm provides a theoretically optimal policy, it is difficult to implement directly in practice due to its dependence on future access sequences. To overcome this limitation, this proposal introduces inverse reinforcement learning (IRL) technology to infer the underlying reward function behind the optimal cache policy and trains the policy network of the cache replacement policy based on this function.

[0152] Distributed Belady's algorithm is considered as the ideal expert strategy π * (a t |S(t),G(t)), and the goal of IRL is to infer a reward function R(s,a) from these expert policies so that the expert policy is optimal under this reward function.

[0153] Optionally, the inverse reinforcement learning algorithm is a maximum entropy inverse reinforcement learning method.

[0154] Optionally, determining the reward function of the cache replacement strategy by an inverse reinforcement learning algorithm includes:

[0155] According to the inverse reinforcement learning algorithm, maximizing the likelihood probability of the cache replacement strategy trajectory is determined as the reward function.

[0156] Alternatively, in the maximum entropy inverse reinforcement learning method, the expert policy is assumed to follow the Boltzmann distribution:

[0157]

[0158] Based on this assumption, the objective function of IRL can be defined as maximizing the likelihood probability of the cache replacement strategy trajectory:

[0159]

[0160] Where D is the set of expert trajectories generated by the distributed Belady's algorithm and λ is the regularization parameter.

[0161] Optionally, training a policy network of the cache replacement strategy according to the reward function to obtain a target cache strategy includes:

[0162] According to the reward function, the policy network is trained using a policy gradient method to determine network parameters of the policy network in the cache replacement strategy;

[0163] The target caching strategy is determined according to the network parameters.

[0164] In the embodiment of the present application, the IRL algorithm decomposes the reward function into a global component and a local component: R(s,a)=Rg (s,a)+R l (s,a);

[0165] Among them, the global component R g captures the impact of network topology and overall data access patterns, while local components R l This decomposition can effectively aggregate information in a distributed environment while protecting node privacy.

[0166] After determining the reward function, the policy network π is trained using the policy gradient method according to the reward function. θ (a t |s i (t)), where θ represents the network parameters, then the goal of the policy gradient is to maximize the expected reward:

[0167]

[0168] The policy gradient update rule can be expressed as Where α is the learning rate, Express expectations.

[0169] In the embodiment of the present application, in order to further conform to the characteristics of the distributed environment, two additional loss terms are introduced: a reusability loss term and a spectral regularization term based on the network structure;

[0170] Among them, the reusability loss term L reuse (θ) encourages the system to more accurately predict the reuse distance of data:

[0171]

[0172] in, represents the reuse distance predicted by the policy network, r i (t) is the actual reuse distance. Represents the expected value operator, all possible states s i (t) and action a t Take the average, these states and actions are based on the current policy π θ obtained by sampling;

[0173] The spectral regularization term based on the network structure encourages adjacent nodes to adopt similar strategies:

[0174]

[0175] According to the reusability loss term and the network structure-based spectral regularization term, the final optimization goal is determined as:

[0176] max θ J(θ)-λ1Lreuse (θ)-λ2L spectral (θ)

[0177] Among them, λ1 and λ2 are hyperparameters that weigh different objectives.

[0178] The data caching method of the present application embodiment, through an inverse reinforcement learning-based approach, not only enables the system to learn a near-optimal cache replacement strategy but also provides a deep understanding of the decision-making logic behind the strategy, enabling the system to make intelligent and adaptable decisions when faced with new and unseen network structures and data access patterns. Furthermore, training the policy network through the reward function, rather than direct policy mapping, offers improved generalization and flexibility, enabling it to better cope with the dynamic changes and uncertainties in distributed environments.

[0179] The data caching method of the embodiment of the present application is as follows Figure 3 As shown:

[0180] The system data of the distributed cache system is acquired through the input layer, specifically including: network topology information, real-time status of each node and data access mode in the system;

[0181] The system data is abstracted into a dynamic graph structure through the modeling layer, and spectral graph embedding data reflecting the network topology characteristics is generated by calculating the Laplace matrix and spectral decomposition. It can effectively capture the relationship between nodes and the global structure of the network, and convert each node into a position coordinate representation.

[0182] By introducing an attention mechanism at the representation layer, each node can perceive its own and surrounding states, and has a comprehensive information perspective when making decisions. The spectral attention mechanism aggregates important information in the network by calculating the similarity and positional relationships between nodes, thereby improving decision-making accuracy and information utilization efficiency.

[0183] The distributed Belady's algorithm is adopted as the ideal expert strategy at the decision-making layer, and inverse reinforcement learning technology is used to infer the potential reward function behind the optimal caching strategy. Through the maximum entropy inverse reinforcement learning method, the system observes the decision trajectory of the distributed Belady's algorithm and infers the reward function that can explain these optimal decisions. Based on the inferred reward function, the system uses reinforcement learning methods to train the policy network of each node. This enables the system to learn near-optimal cache replacement strategies and deeply understand the decision-making logic behind the strategies, so as to make intelligent and adaptable decisions when faced with new network structures and data reuse patterns.

[0184] The above describes various methods of the embodiments of the present application. The following further provides apparatuses for implementing the above methods.

[0185] like Figure 4 As shown, the embodiment of the present application further provides a data cache device 400, including:

[0186] A generating module 401 is configured to generate spectrogram embedded data based on the obtained system data of the distributed cache system; the spectrogram embedded data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system;

[0187] A first determination module 402 is configured to determine, based on the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for representing content similarity and position similarity between the nodes according to an attention mechanism; the content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes;

[0188] The second determination module 403 is used to determine a cache replacement strategy based on the spectrum embedding vector and the access information stored by each node in the distributed cache system; the cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

[0189] The data caching device provided in the embodiments of the present application generates spectral graph embedding data from acquired distributed cache system system data. Based on the generated spectral graph embedding data, the device determines the relationships between nodes and the global characteristics of the distributed cache system's network topology. The spectral graph embedding vectors are used to determine the similarities and positional relationships between computing nodes, improving decision-making accuracy. The spectral graph embedding vectors are combined with the access information stored by each node to determine the optimal cache replacement strategy. This addresses the problem in existing technologies that ignores the impact of network topology on data access efficiency, resulting in inaccurate cache decisions.

[0190] like Figure 5 As shown, the embodiment of the present application further provides a data cache device 500, including: a processor 501;

[0191] The processor 501 is configured to generate spectrogram embedded data based on the acquired system data of the distributed cache system; the spectrogram embedded data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system;

[0192] Determining, using the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for characterizing the content similarity and position similarity between the nodes according to an attention mechanism; the content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes;

[0193] A cache replacement strategy is determined based on the spectral graph embedding vector and the access information stored by each node in the distributed cache system; the cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

[0194] like Figure 6 As shown, an embodiment of the present application also provides a network device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the computer program is executed by the processor 601, the computer program implements the various processes of the above-mentioned data caching method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0195] The present application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-mentioned data caching method embodiment and achieves the same technical effect. To avoid repetition, the details are not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0196] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned data caching method embodiment are implemented and can achieve the same technical effect. To avoid repetition, they will not be repeated here.

[0197] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0198] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0199] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A data caching method, characterized in that: include: Generate spectrogram embedding data based on the obtained system data of the distributed cache system; The spectrum embedded data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system; Determining, by using the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for representing the content similarity and position similarity between the nodes according to an attention mechanism; The content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes; determining a cache replacement strategy based on the spectral graph embedding vector and access information stored by each node in the distributed cache system; The cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

2. The data caching method according to claim 1, wherein: Generate spectrum embedding data based on the obtained distributed cache system data, including: Determining a dynamic graph corresponding to the distributed cache system, wherein the dynamic graph is used to represent a relationship between communication links or data transmission paths between nodes that changes over time; Constructing a Laplace matrix of the distributed cache system according to the dynamic graph; Perform spectral decomposition on the Laplace matrix to generate the spectral graph embedding data.

3. The data caching method according to claim 1, wherein: The system data includes: The network topology information of the distributed cache system, the real-time status information of each node, and the data access mode of the distributed cache system; The network topology information is used to indicate the connection relationship between the nodes in the distributed cache system; The real-time status information includes the storage capacity, computing resources, current cache content and load status of each node; The data access pattern includes the frequency and order in which data is accessed in the distributed cache system.

4. The data caching method according to claim 1, wherein: Determining, by using the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for representing the content similarity and position similarity between the nodes according to an attention mechanism, including: Determine a state vector based on the spectrogram embedded data and the state information; the state vector includes: node access data, cache content, access history, and a correspondence between the spectrogram embedded data and time; Determining a spectral graph attention function based on the state vector and the attention mechanism; the spectral graph attention function is related to the content similarity and position similarity between each node; The spectrogram embedding vector is determined according to the spectrogram attention function.

5. The data caching method according to claim 1, wherein: Determining a cache replacement strategy based on the spectral graph embedding vector and access information stored by each node in the distributed cache system includes: Determining access status and importance of stored data items in the distributed cache system using Belady's algorithm based on the spectral graph embedding vector and the access information stored in each node; A cache replacement strategy including eliminated data items is determined based on access conditions and importance of data items stored in the distributed cache system.

6. The data caching method according to claim 1, wherein: The method further comprises: Determining a reward function for the cache replacement strategy using an inverse reinforcement learning algorithm; Training the policy network of the cache replacement strategy according to the reward function to obtain a target cache strategy; the policy network is used to represent the corresponding relationship between the data to be cached and the node state; According to the target cache strategy, a cache scheme for the data to be cached in each node is determined.

7. The data caching method according to claim 6, characterized in that: The reward function of the cache replacement strategy is determined by an inverse reinforcement learning algorithm, including: According to the inverse reinforcement learning algorithm, maximizing the likelihood probability of the cache replacement strategy trajectory is determined as the reward function.

8. The data caching method according to claim 6, wherein: The policy network of the cache replacement strategy is trained according to the reward function to obtain a target cache strategy, including: According to the reward function, the policy network is trained using a policy gradient method to determine network parameters of the policy network in the cache replacement strategy; The target caching strategy is determined according to the network parameters.

9. A data cache device, characterized in that: include: A generation module, used for generating spectrogram embedding data according to the obtained system data of the distributed cache system; The spectrum embedded data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system; A first determination module is configured to determine, based on the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for representing the content similarity and position similarity between the nodes according to an attention mechanism; The content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes; A second determination module is configured to determine a cache replacement strategy based on the spectrum embedding vector and the access information stored by each node in the distributed cache system; The cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

10. A data cache device, characterized in that: include: processor, The processor is configured to generate spectrogram embedding data based on the acquired system data of the distributed cache system; The spectrum embedded data is used to represent the connection relationship between the nodes in the distributed cache system and the position coordinates of each node in the distributed cache system; Determining, by using the spectrogram embedding data and the state information of the nodes in the system data, a spectrogram embedding vector for representing the content similarity and position similarity between the nodes according to an attention mechanism; The content similarity is used to indicate the similarity of the states between the nodes, and the position similarity is used to indicate the distance between the positions of the nodes; determining a cache replacement strategy based on the spectral graph embedding vector and access information stored by each node in the distributed cache system; The cache replacement strategy is used to characterize the state set of each node and the topology of the distributed cache system.

11. A network device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program implements the steps of the data caching method according to any one of claims 1 to 8 when executed by the processor.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the data caching method according to any one of claims 1 to 8.

13. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the data caching method according to any one of claims 1 to 8.

Images