Data processing method, device, equipment and storage medium

Abstract

The present application discloses a data processing method, device, equipment and storage medium, belonging to the field of computer technology. The method includes constructing a weighted directed self-loop-free graph of multi-user data access including M first nodes and a first directed connection edge connecting every two first nodes based on data access logs of N users accessing data objects, wherein the first node is the i-th accessed data object of at least one user; extracting P sampling subgraphs from the weighted directed self-loop-free graph; based on each sampling subgraph and its corresponding label vector, training a first graph neural network and a pre-fetch decoder model, and obtaining a second graph neural network and a pre-fetch decoder for generating a probability value of a user accessing P data objects with an associated relationship in a second time period based on the data access log of the user in the first time period. In this way, the data object to be accessed is determined by learning the associated relationship of the accessed data through the graph neural network and the decoder, so as to improve the hit rate of cache pre-fetching.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to a data processing method, device, equipment and storage medium. Background Art

[0002] Cache prefetching is loading data or resources from a database or disk file into the cache before they are actually needed, achieving temporary storage, avoiding frequent access to the database or disk file, and improving system response speed and performance. Cache prefetching can be rule-based, that is, manually setting rules to guide cache operations, that is, performing cache prefetching operations when the cache status meets the rules.

[0003] However, as data center applications have increased their demand for high performance and scalability, data access patterns have gradually evolved to be multi-user, long-term, and complex. A large number of users have different access requirements, and access rules and patterns are no longer distinctly periodic, regular, and frequent. Therefore, since the aforementioned cache prefetching cannot take into account multi-user, long-term, and complex access patterns, the cache prefetching hit rate is low, resulting in frequent access to databases or disk files, reducing system response speed and performance. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, apparatus, device and storage medium, which can solve the problem of low cache prefetch hit rate in the related art.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, which may include:

[0006] Obtain data access logs of N users accessing data objects, where N is an integer greater than 1;

[0007] Based on the data access log, a weighted directed graph without self-loops of multi-user data access is constructed, wherein the weighted directed graph without self-loops includes M first nodes and a first directed connection edge connecting every two first nodes among the M first nodes, wherein the first node is the i-th accessed data object of at least one user among the N users, and the first directed connection edge is used to indicate the importance of the order of at least one user's request to access the data object from one first node connected by the first directed connection edge to another first node in the order of N users accessing the M first nodes in the data access log, wherein M is an integer greater than 1, and i∈[1,M];

[0008] Extracting P sampling subgraphs from a weighted directed graph without self-loops, the sampling subgraphs comprising a second node and a second directed connecting edge connecting a path where the second node is located, the second node being a node whose node out-degree is greater than or equal to a preset node out-degree among the M first nodes, and P is an integer greater than 1;

[0009] Based on each sampling subgraph in the P sampling subgraphs and the label vector corresponding to each sampling subgraph, the first graph neural network and pre-fetch decoder model are trained until the preset training conditions are met to obtain the second graph neural network and pre-fetch decoder, which are used to generate a probability value of the target user accessing the data object with the associated relationship in the second time period based on the data access log of the target user in the first time period.

[0010] In a second aspect, an embodiment of the present application provides a data processing device, which may include:

[0011] An acquisition module is used to acquire data access logs of N users accessing data objects, where N is an integer greater than 1;

[0012] A construction module is used to construct a weighted directed graph without self-loops of multi-user data access based on a data access log, wherein the weighted directed graph without self-loops includes M first nodes and a first directed connection edge connecting every two first nodes among the M first nodes, wherein the first node is the i-th accessed data object of at least one user among the N users, and the first directed connection edge is used to indicate the importance of the order of at least one user's request to access the data object from one first node connected to the first directed connection edge to another first node in the data access log, i∈[1,M], where M is an integer greater than 1;

[0013] An extraction module is used to extract P sampling subgraphs from the weighted directed self-loop-free graph, where the sampling subgraphs include a second node and a second directed connection edge that concatenates the path where the second node is located, where the second node is a node whose node out-degree is greater than or equal to a preset node out-degree among the M first nodes, and P is an integer greater than 1;

[0014] A training module is used to train the first graph neural network and pre-fetch decoder model based on each sampling subgraph in P sampling subgraphs and the label vector corresponding to each sampling subgraph until the preset training conditions are met to obtain the second graph neural network and pre-fetch decoder. The second graph neural network and pre-fetch decoder are used to generate a probability value of the target user accessing the data object with the associated relationship in the second time period based on the data access log of the target user in the first time period.

[0015] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory storing computer program instructions;

[0016] When the processor executes the computer program instructions, the data processing method shown in the first aspect is implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the data processing method shown in the first aspect is implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the data processing method shown in the first aspect.

[0019] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the data processing method shown in the first aspect.

[0020] The data processing method, apparatus, device and storage medium of the embodiments of the present application can construct a weighted directed graph without self-loops of multi-user data access based on a data access log of N users accessing a data object, wherein the weighted directed graph without self-loops includes M first nodes and a first directed connection edge connecting every two first nodes among the M first nodes, wherein the first node is the i-th data object accessed by at least one user among the N users, and the first directed connection edge is used to indicate the importance of the order of at least one user requesting to access the data object from one first node connected by the first directed connection edge to another first node in the data access log to the order of N users accessing the M first nodes; then , extract P sampling subgraphs from a weighted directed graph without self-loops, the sampling subgraphs including a second node and a second directed connection edge connecting the path where the second node is located, the second node being a node whose node out-degree is greater than or equal to a preset node out-degree among the M first nodes; based on each sampling subgraph in the P sampling subgraphs and a label vector corresponding to each sampling subgraph, train the first graph neural network and pre-fetch decoder model until the preset training conditions are met, and obtain a second graph neural network and a pre-fetch decoder, the second graph neural network and the pre-fetch decoder are used to generate a probability value of a target user accessing a data object with an associated relationship in a second time period based on the data access log of the target user in the first time period. In this way, the data access log can be modeled as a weighted directed graph without self-loops, effectively retaining the data access pattern of a single user, such as access order, access frequency, etc., effectively modeling a longer access sequence, reducing redundant information, and using graph neural networks and prefetch decoder models to learn the correlation of access data. It can effectively model the access patterns of multiple users, long sequences, and multiple sessions, adapt to the dynamic changes of data, and adapt to the data access scenarios of multiple users. While retaining the access data of a single user in the multi-user access scenario, the graph neural network and prefetch decoder model can capture more complex access data than traditional sequence data, encode more information containing Embedded vectors, learn access pattern representations in multi-user scenarios, and thus more accurately predict the user's data access in the next time period, reduce the number of cache misses and delays, improve the cache prefetch hit rate, and reduce frequent access to databases or disk files, thereby improving system response speed and performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of generating a data access log sequence for a data processing method provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of constructing a weighted directed graph without self-loops according to a data processing method provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of determining a second node in a data processing method provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of constructing a sampling subgraph of a data processing method provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of constructing an access subgraph for a data processing method provided in an embodiment of the present application;

[0028] Figure 7 The intention of constructing an output vector representation of a data processing method provided in an embodiment of the present application;

[0029] Figure 8 One of the schematic diagrams of a node structure of a data processing method provided in an embodiment of the present application;

[0030] Fig. 9 A second schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0031] Fig.10 A third schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0032] Fig.11 A fourth schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0033] Fig.12 A fifth schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0034] Fig.13 A sixth schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0035] Fig.14 A seventh schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0036] Fig.15 An eighth schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0037] Fig.16A ninth schematic diagram of a node structure of a data processing method provided in an embodiment of the present application;

[0038] Fig.17 is a structural schematic diagram of a data processing device provided by an embodiment of the present application;

[0039] Fig.18 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application;

[0040] Fig.19 A schematic diagram of the hardware structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0041] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0042] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0043] The acquisition, storage, use, and processing of data (including but not limited to the features and information in the text) in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. In order to solve the above technical problems, the embodiments of this application provide a data processing method, device, computer equipment, and storage medium.

[0044] Cache prefetching is a strategy used to optimize system performance and user experience. It loads data or resources from a database or disk file into the cache before they are actually needed. This improves system response speed and performance by avoiding frequent access to the database or disk file through fast temporary storage. In addition, the cache can also temporarily store calculation results, reduce repeated calculations, and achieve multi-server load balancing.

[0045] In related technologies, cache prefetching can generally be divided into two categories: rule-based cache prefetching and machine learning-based cache prefetching. The former guides the operation of the cache by manually setting rules, and performs cache prefetching operations only when the state of the cache satisfies these rules. The latter adopts a data-driven approach, using machine learning or deep learning technology to mine the laws and patterns of data access to manage the cache more intelligently. These methods may use machine learning algorithms such as decision trees and support vector machines to predict which data will be frequently accessed by analyzing the data access patterns of the application, and adjust the weights of the prefetching algorithm accordingly, thereby realizing the prediction of future data so that the subsequent data objects of the linear sequence can be stored in the cache.

[0046] As data center applications have increasingly demanded high performance and scalability, data access patterns have gradually evolved toward multi-user, long-term, and complex characteristics. A large number of users may have different access requirements, and access patterns are no longer distinctly periodic, regular, and frequent. Since the above two methods cannot adapt to changes in data access patterns, they need to rely on manually designed rules, ignore complex patterns, and need to constantly redesign rules, wasting human resources while reducing rule production efficiency; or abstract all access objects into linear sequences, ignoring the complex, nonlinear sequence access patterns that may appear during the access process; or only consider single-user short-term patterns, ignoring multi-user long-term pattern information, similar access patterns of similar users, and irreversible request orders, etc., which require access patterns modeled with directed graph structures. Therefore, in the data modeling process, the directed graph structure formed by the interweaving of multiple user data access sequences is converted into a linear, sequential structure with only a sequential order, which cannot consider multi-user, long-term, and complex access patterns, which easily leads to a low cache prefetch hit rate, resulting in frequent access to databases or disk files, and reducing system response speed and performance.

[0047] Therefore, an embodiment of the present application provides a data processing method, which can effectively model the access patterns of multiple users, long sequences, and multiple sessions by modeling the data access logs of multiple users as a weighted directed graph structure that can preserve the access characteristics of multiple users, and use a graph neural network to learn the correlation of data access and the data access pattern in multi-user scenarios. The graph embedding is then decoded by a decoder to predict the user's data access at the next moment in combination with the access patterns and historical access patterns of other users, and finally the predicted data objects are pre-fetched into the cache. In this way, the correlation between data accesses is learned through the graph neural network, and the data access pattern in the multi-user system can be learned, and data with a high probability of subsequent access can be predicted. If the data is not currently in the cache, the data is loaded into the cache, which can improve the cache hit rate.

[0048] Based on this, the following will combine the attached Figures 1 to 19 , describes in detail the data processing method, device, computer equipment and storage medium of the embodiments of the present application. It should be noted that these embodiments are not intended to limit the scope of the disclosure of the present application.

[0049] In order to better illustrate the contents in the embodiments of the present application, the following are respectively combined with Figure 1 A data processing method provided in an embodiment of the present application is described as follows.

[0050] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application.

[0051] like Figure 1 As shown, the data processing method can be applied to the computer device in the embodiment of the present application, and the data processing method can specifically include the following steps:

[0052] Step 110, obtaining data access logs of N users accessing data objects, where N is an integer greater than 1; Step 120, constructing a weighted directed graph without self-loops of multi-user data access based on the data access logs, wherein the weighted directed graph without self-loops includes M first nodes and a first directed connection edge connecting every two first nodes among the M first nodes, wherein the first node is the i-th accessed data object of at least one user among the N users, and the first directed connection edge is used to indicate the importance of the order of at least one user requesting to access the data object from one first node connected to the first directed connection edge to another first node in the data access log, wherein M is an integer greater than 1, i∈[1,M]; Step Step 130, extracting P sampling subgraphs from a weighted directed graph without self-loops, the sampling subgraphs including a second node and a second directed connecting edge of a path in which the second node is located, the second node being a node whose node out-degree is greater than or equal to a preset node out-degree among the M first nodes, and P being an integer greater than 1; Step 140, training a first graph neural network and a pre-fetch decoder model based on each sampling subgraph in the P sampling subgraphs and a label vector corresponding to each sampling subgraph until preset training conditions are met, and obtaining a second graph neural network and a pre-fetch decoder, the second graph neural network and the pre-fetch decoder being used to generate a probability value of a target user accessing a data object having an associated relationship in a second time period based on the data access log of the target user in a first time period.

[0053] In this way, the data access log can be modeled as a weighted directed graph without self-loops, effectively retaining the data access pattern of a single user, such as access order, access frequency, etc., effectively modeling a longer access sequence, reducing redundant information, and using graph neural networks and prefetch decoder models to learn the correlation of access data. It can effectively model the access patterns of multiple users, long sequences, and multiple sessions, adapt to the dynamic changes of data, and adapt to the data access scenarios of multiple users. While retaining the access data of a single user in the multi-user access scenario, the graph neural network and prefetch decoder model can capture more complex access data than traditional sequence data, encode more information containing Embedded vectors, learn access pattern representations in multi-user scenarios, and thus more accurately predict the user's data access in the next time period, reduce the number of cache misses and delays, improve the cache prefetch hit rate, and reduce frequent access to databases or disk files, thereby improving system response speed and performance.

[0054] The above steps are described in detail below, as shown below.

[0055] First, in step 120, the data access log can be modeled as a weighted directed graph without self-loops. This process can include extracting data access log sequences, constructing a weighted directed graph without self-loops of single-user data access, and constructing a weighted directed graph without self-loops of multi-user data access.

[0056] Based on this, in some embodiments of the present application, step 120 may specifically include step 1201 and step 1202.

[0057] Step 1201, based on the data access log, construct a data access log sequence set, the data access log sequence set includes the data access log sequence of each user among N users accessing the data object, the data access log sequence includes an ordered set of data objects requested to be accessed by the user in the same session, N is an integer greater than 1, i∈[1,N].

[0058] Furthermore, step 1201 may specifically include step 12011 and step 12012.

[0059] Step 12011: extract the data access log sequence of each user among N users from the data access log in units of sessions.

[0060] For example, Figure 2 As shown, data access log sequences d1, d2, d3, d4, ... are extracted from data access log D, and data access log sequence D of a single user is extracted from data access log D in units of sessions. x ={d x1 , d x2 , ..., d xy}, for example, D1{d 1，1 , d 1，3 , ..., d 1y}, D2{d 2，1 , d 2，3 , ..., d 2y}, where d xy Represents the yth requested data pair of the xth user in the data access log, where x∈[1,N] and y is a positive integer.

[0061] Step 12012, summarizing the data access log sequence of each user among the N users to obtain a data access log sequence set.

[0062] For example, a single user data access log sequence D x It is an ordered set of request data of the x-th user in a session or service in the data access log sequence set. The data access log sequence set D is the set of all single user data access log sequences, D = {D1, D2, ..., D N}.

[0063] Step 1202: construct a weighted directed graph without self-loops of multi-user data access based on a data access log sequence of each user among the N users accessing the data object.

[0064] For example, for each single-user data access log sequence D in the data access log sequence set D x , generate a weighted directed graph without self-loops g x , the node data of the first node is the sequence D of the data access log x Each data d xy .

[0065] Furthermore, step 1202 may specifically include step 12021 and step 12022.

[0066] Step 12021, based on the data access log sequence of the jth user among N users accessing the data object, generate a first weighted directed graph without self-loops for the jth user, wherein the first weighted directed graph without self-loops includes K third nodes and a third directed connection edge connecting every two third nodes among the K third nodes, the K third nodes are data objects requested to be accessed by the jth user in the same session, the third directed connection edge is used to indicate the importance of the order of the jth user from one third node connected by the third directed connection edge to another third node in the data access log sequence of the jth user accessing the data object, and j∈[1,N], K is an integer greater than 1.

[0067] For example, Figure 3 As shown, the single user data access log sequence D i , i∈[1,N] can be the data access log sequence of the jth user accessing the data object, the weighted directed non-self-loop graph g i It can be the first weighted directed graph without self-loops of the j-th user.

[0068] Step 12022, merge the second weighted directed graphs with no self-loops of the users except the j-th user among the N users into the first weighted directed graph with no self-loops to obtain a weighted directed graph with no self-loops for multi-user data access.

[0069] For example, to construct a weighted directed graph without self-loops for multi-user data access, we can still refer to Figure 3 , the first weighted directed self-loop-free graph g of the jth user iand the original multi-user data access weighted directed self-loop graph G (the multi-user data access weighted directed self-loop graph G can be the second weighted directed self-loop graph of one user except the j-th user among the N users, or can be the weighted directed self-loop graph after the second weighted directed self-loop graph of each user among at least two users except the j-th user is fused). Here, if there are added nodes, they can be added to the fused weighted directed self-loop graph of the multi-user data access. If there are repeated nodes, they can be not added to the fused weighted directed self-loop graph of the multi-user data access, and their nodes can be reused. Only directed connection edges are added. If the directed connection edge already exists, the weight of the importance represented by the directed connection edge in the fused weighted directed self-loop graph of the multi-user data access is updated to its original value and g. x The sum of the edge weights (i.e., the sum of the weights of at least two repeated directed connected edges) is used to construct a weighted directed self-loop-free graph of single-user data access and a weighted directed self-loop-free graph of multi-user data access from a single data access log sequence.

[0070] Furthermore, regarding step 130, in some embodiments of the present application, each first node in the weighted directed graph without self-loops corresponds to a node out-degree, and the node out-degree is used to characterize the number of edges of the first node in the weighted directed graph without self-loops with the first node as the tail; the P sampling subgraphs include a sampling subgraph of each second node in the P second nodes, based on this, the step 130 can specifically include steps 1301 to 1303.

[0071] Step 1301: Randomly select P second nodes whose node out-degree is greater than or equal to a preset node out-degree from M first nodes in a weighted directed graph without self-loops, where P is an integer greater than 1.

[0072] For example, Figure 4 As shown, select nodes, randomly select nodes with degree greater than 0 from the multi-user data access weighted directed non-self-loop graph G, and record the P second nodes selected by this method = {P1, P2, ..., P P For example, P is 4, that is, 4 second nodes = d7, d8, d 10 , d 16 .

[0073] Step 1302, based on each second node among the P second nodes, extract the previous second node R hops before each second node, the successor second node after each second node, and the second directed connection edge connecting each second node, the previous second node and the successor second node.

[0074] Step 1303, generates a sampling subgraph of each second node based on each second node, the second node before each second node R hops, the successor second node of each second node, and the second directed connection edge connecting each second node, the second node before each second node, and the successor second node.

[0075] For example, Figure 5 As shown, for the node set, the second node = {d7, d8, d 10 , d 16 Each second node n in i , sample its previous hop second node, the next hop second node and the edges between them, and compare them with the selected second node n i Together they form a sampling subgraph S in For example, the second node d7, R is 2, that is, the second node of the first 2 hops can be: d 14 →d4,d 14 →d 15 , d2→d4, d 15 →d 16 , d1→d4, the second successor node can be d 17 , d 11 , d 10 Based on this, the sampling subgraph s2 constructed by the second node d7. In this way, for the second node = {d7, d8, d 10 , d 16 After executing this step for all the second nodes of}, we get the sampling subgraph = {S1, S2, ..., S P}.

[0076] Then, involving step 140, in some embodiments of the present application, before step 140, a label vector of each sampling subgraph can be calculated, based on which the P sampling subgraphs include each sampling subgraph in the P sampling subgraphs and a label vector corresponding to each sampling subgraph, and the data processing method can also include steps 1401 and 1402.

[0077] Step 1401, divide the Qth sampling subgraph among the P sampling subgraphs to obtain the input subgraph and input label vector of the Qth sampling subgraph, the sampling subgraph includes the target node in the sampling subgraph, the target previous hop node T hops before the target node, and the first target directed connection edge connecting the target node and the target previous hop node in series, and the input label vector is the normalized vector of the second target directed connection edge connecting the target node and the target successor node of the target node in series, T∈[1,R], Q∈[1,P].

[0078] For example, Figure 6 As shown, a sampling subgraph-label vector is constructed, that is, each sampling subgraph in the sampling subgraph set is divided into an input sampling subgraph S inand label vector T. Among them, S in is the node n i 、n i The front j in A directed weighted graph of hop nodes and their edges; T = {t1, t2, ..., t M} is to put node n i The normalized vector of the directed edge weights of its successor nodes, whose length M is the number of data objects available for pre-fetching in the file system. .

[0079] Step 1402, calculate the label vector of the Qth sampling subgraph based on the target node, the target successor node, the source node and the end node in the Qth sampling subgraph, and the first directed connection edge from the source node to the end node in the weighted directed acyclic graph. The label vector is used to characterize the probability that the node corresponding to the data object in the data access log is the target successor node after the target node.

[0080] For example, when a data object is not a node n i When the successor node is , the operation result is recorded as 0, and its probability is calculated as follows:

[0081] (1)

[0082] Among them, N next is a node, n i The set of successor nodes, n from , n to are the source node and the terminal node respectively, and w is the multi-user data access weighted directed non-self-loop graph G in n from to n to The weight of the directed edge.

[0083] Based on this, in some embodiments of the present application, the first graph neural network and pre-fetch decoder model include a first graph neural network model and a first pre-fetch decoder model, and step 140 may specifically include steps 1401 to 1403. Step 1401, encode each sampling subgraph through the first graph neural network model to obtain a first representation vector for representing the data access pattern.

[0084] Step 1402: decode the first representation vector using a first pre-fetch decoder model to obtain a first probability value of a user requesting to access a data object in a third time period.

[0085] Step 1403, when the cross entropy loss value between the vector of the first probability value and the label vector is less than or equal to the preset cross entropy loss value, it is determined that the preset training condition is met, and a second graph neural network and a pre-fetch decoder are obtained.

[0086] Exemplarily, the first graph neural network and pre-fetch decoder are trained to learn the data access relevance. The graph neural network and pre-fetch decoder are trained using the sampled subgraph-label vector to learn the data access relevance. GNN As an encoder, the input sampling subgraph S in Encoded into a representation vector E representing the data access pattern encode ; Prefetch decoder f decode To represent the vector E encode is the input, which is decoded into the probability E of the user requesting a file in the file system at the next moment prod To minimize E prod The cross entropy loss L with the label vector T prefetch Update the graph neural network f for the target GNN and prefetch decoder f decode parameters to enable it to learn the association of data access. prefetch The calculation formula (2) is as follows:

[0087] (2)

[0088] Where k is the number of sampled subgraph-label vectors; M is the dimension of the label vector, which is equal to the number of files available for pre-fetching in the file system; t ij Represents the value of the jth dimension of the i-th label vector; E ij prod Represents the probability of the data object of the j-th replaceable file in the file system being accessed at the next moment after the i-th sampling subgraph passes through the graph neural network and prefetch decoder.

[0089] In addition, in some embodiments of the present application, after step 140, the data processing method provided by the embodiment of the present application further provides a weighted directed graph with no self-loops and a second weighted directed graph with no self-loops based on the multi-user data access determined above, and filters the data objects stored in the cache. Based on this, the user request data at the next moment can be predicted by using a graph neural network and a pre-fetch decoder. The process may include constructing an access subgraph from a real-time access sequence, enhancing the access subgraph using a weighted directed graph with no self-loops for multi-user data access, and using a graph neural network for encoding and a pre-fetch decoder for decoding to obtain the probability of the user requesting the data object at the next moment. The data processing method also includes steps 1501 to 1505.

[0090] Step 1501: Obtain a target data access log sequence of a target user accessing a data object within a first time period.

[0091] Step 1502: Generate a second weighted directed non-self-loop graph of the target user based on the target data access log sequence.

[0092] Furthermore, step 1502 may specifically include steps 15021 to 15023.

[0093] Step 15021, according to the order in which the target user accesses the data objects in the first time period, the data objects accessed by the target user in the target data access log sequence in the first time period are arranged to obtain a data object set. Step 15022, based on the number of times the target user accesses the second target data object after accessing the first target data object in the data object set, a fourth directed connection edge is generated that connects the first access node corresponding to the first target data object and the second access node corresponding to the second target data.

[0094] Step 15023: connect the first target data object and the second target data object in series through the fourth directed connection edge to generate a second weighted directed graph without self-loops of the target user.

[0095] For example, Figure 7 As shown in Figure 1, an access subgraph is constructed from the real-time access sequence. The access sequence R of a single user in a certain period of time is collected. run ={r1,r2,……r c} and construct it into an access subgraph S according to the access order rep . R run The value of should be an existing file in the file system. Access subgraph S rep It is a weighted directed graph, whose node value is the requested data object, and the edges point from the previously accessed data object to the next accessed data object in chronological order. The weight of the edge is the number of times the successor object is accessed after accessing a data object.

[0096] Step 1503, performing graph enhancement processing on the second weighted directed graph without self-loops of the target user through the weighted directed graph without self-loops of multi-user data access to obtain a third weighted directed graph without self-loops.

[0097] Furthermore, step 1503 may specifically include steps 15031 to 15033.

[0098] Step 15031, filter out the target terminal node in the second weighted directed graph without self-loops from the weighted directed graph without self-loops of multi-user data access.

[0099] Step 15032: extract the target previous hop node that is D hops before the target terminal node from the weighted directed self-loop-free graph of multi-user data access to obtain an enhanced subgraph.

[0100] Step 15033, merge the enhanced subgraph and the second weighted directed graph without self-loops to obtain a third weighted directed graph without self-loops.

[0101] Exemplarily, the access subgraph is enhanced using a multi-user data access weighted directed graph without self-loops. Select an access sequence R from the multi-user data access weighted directed graph without self-loops G run The last item r c The first j of the corresponding node in Jump nodes, as an enhanced subgraph S en . The enhancer map S en With access to subgraph s rep After performing the merge operation, the enhanced access subgraph to be encoded S is obtained encode .

[0102] Step 1504: Process the third weighted directed non-self-loop graph through a second graph neural network and a pre-fetch decoder to obtain a probability value of the target user accessing a data object associated with the target data access log sequence in the second time period.

[0103] Exemplarily, the process of constructing an access subgraph from a real-time access sequence and enhancing the access subgraph using a multi-user data access weighted directed non-self-loop graph is as follows: Figure 8 As shown in the figure, the graph neural network is used for encoding and the pre-fetch decoder is used for decoding to obtain the probability of the user requesting the data object at the next moment. encode Input graph neural network f GNN , get the encoded access subgraph S encode The characterization vector E encode ; Prefetch decoder f decode To represent the vector E encode is the input, and decoded into the output vector E of the probability of the user requesting a file in the file system at the next moment prob , whose length M is the number of data objects available for prefetching in the file system.

[0104] Step 1505 , based on the probability value of the target user accessing the data object associated with the target data access log sequence in the second time period, the target data object is screened and stored in the cache, and the probability value of the target data object is greater than or equal to a preset probability value.

[0105] Exemplarily, the predicted data is stored in a cache. Select the output vector E prob The data objects corresponding to the top i high values ​​are read from the file system and stored in the cache.

[0106] Therefore, the embodiment of the present application provides a data processing method for cache prefetching based on log process representation learning, including modeling the data access log as a weighted directed graph without self-loops; constructing a sampling subgraph set through sampling; learning the correlation of data access through a graph neural network, and encoding it into an embedding vector containing an access pattern; a decoder decodes the embedding vector to obtain the probability of the next data access; and prefetching data objects based on the data access probability. Specifically, based on the single-user data access model of the weighted directed graph without self-loops structure, the weighted directed graph without self-loops structure can effectively retain the data access pattern information of a single user, such as access order, access frequency, etc., and can effectively model longer access sequences and reduce redundant information. The multi-user data access model based on the weighted directed graph structure can effectively model the access patterns of multiple users, long sequences, and multiple sessions, and can adapt to the dynamic changes of data. It is constructed from the single-user data access model, so it can adapt to the access scenarios of multiple users, and can retain the single-user access pattern information in the multi-user access scenario while constructing the associated information of the multi-user access pattern, which retains more data access pattern information than the traditional multi-user data access modeling method. The data prefetching method based on the graph neural network can capture more complex access pattern information than traditional sequence data, encode more information containing embedded vectors, learn access pattern representations in multi-user scenarios, and more accurately predict the data that may be requested next, thereby improving the cache hit rate.

[0107] In order to explain the data processing method of the present application in detail, the following is explained using a cache system as an example.

[0108] This embodiment takes a cache system as an example, and takes the access log "access.log" cached for a period of time, which contains the ID of the user who initiated the access, the time of initiation of the access, and the target object of the access. In this embodiment, there are 20,000 log records in total, and 5,000 independent objects are requested in total. The data processing method may include S101 to S111.

[0109] S101, read the access log file "access.log" and extract the data access log sequence. Extract the single user data access log sequence D from the log in units of sessions. i ={d i，1 , d i，2 , ..., d i，j A data access log sequence is an ordered set of request data representing a user in a service. In this embodiment, a total of 1600 log access sequences are extracted, denoted as D={D1, D2, ..., D 1600 Taking D1 as an example, D1={d3, d 28 , d 24 , d15 , d 66 , d 45 , d 96 , d 66 , d 21}.

[0110] S101, construct a single-user weighted directed non-self-loop graph.

[0111] Take D1={d3,d 28 , d 24 , d 15 , d 66 , d 45 , d 96 , d 66 , d 21} as an example, the result of converting it into a weighted directed graph without self-loops g1 is as follows Fig. 9 As shown:

[0112] Based on the above steps, D={D1, D2, ..., D 1600 Each log access sequence D1 in} is transformed into a weighted directed non-self-loop graph g i .

[0113] S103, construct a multi-user data access weighted directed self-loop-free graph G. The multi-user data access weighted directed self-loop-free graph G is initially an empty graph without nodes and edges. i With G, we can get the constructed multi-user data access weighted directed non-self-loop graph G. Taking G that has been spliced ​​with g1 as an example, the process of splicing it with g2 is as follows Fig.10 As shown:

[0114] S104, select P second nodes {P1, P2, ..., P P In this step, it is assumed that the local Fig.11 As shown, the P second nodes P1, P2, and P3 are d3, d 15 , d 49 .

[0115] S105, for P second nodes {P1, P2, ..., P P Each second node P1 in}, samples its previous j in Jump nodes, successor nodes and the edges between them, and associate them with the selected node P P Together they form a sampling subgraph S in For example, for P1, P2, and P3, we select the first two hops of the nodes, and the obtained sampling subgraphs S1, S2, and S3 are as follows: Fig.12 Shown

[0116] S106, construct the sampling subgraph-label vector. Taking S2 as an example, the label vector calculation result is: T = [0, ..., 0.4, ..., 0.6, ..., 0], whose length is the number of independent objects in this example, 5000, and its 17th and 66th bit values ​​are 0.4 and 0.6 respectively, that is, node d 15 The result after normalizing the weights of the directed edges with its successor nodes.

[0117] Taking S2 as an example, the transformed input subgraph S in and label vector T such as Fig.13 shown.

[0118] S107, train the graph neural network and pre-fetch decoder to learn data access association. GNN Take the input subgraph S in is the input and is encoded into a representation vector E encode ; Prefetch decoder f decode To represent the vector E encode is the input, which is decoded into the probability E of the user requesting a file in the file system at the next moment prod Taking S2 as an example, the encoded E prod =[0.01, …, 0.02, 0.29, …, 0.66, …, 0.01], and T = [0, …, 0.4, …, 0.6, …, 0] to calculate the cross entropy loss -0.32334. In this way, all sampled subgraphs are used with the graph neural network f GNN Encoding and pre-fetch decoder f decode After decoding, the corresponding label vector is used to calculate the cross entropy loss L prefetch And add and average. To minimize L prefetch As the goal, optimize the graph neural network f GNN and prefetch decoder f decode In this embodiment, the termination condition of the training is L prefetch <0.125 or the number of repeated optimization is greater than 40 times.

[0119] S108: construct an access subgraph from the real-time access sequence. run ={d 37 , d 61 , d 55 , d 54 , d 70 , d 15}. This access sequence is constructed as an access subgraph S req ,like Fig.14 shown.

[0120] S109, such as Fig.15As shown, the access sequence R is selected from the multi-user data access weighted directed non-self-loop graph G run The last item d 15 The first two hops of the corresponding node are used as the enhanced subgraph S en . The enhancer map S en With access to subgraph S req After performing the merge operation, the enhanced access subgraph to be encoded S is obtained encode .

[0121] S110, access the subgraph S to be encoded encode Input graph neural network f GNN , get the representative encoding access subgraph S encode Characterization vector E of multi-user data access characteristics encode ; Prefetch decoder f decode To represent the vector E encode is the input, and decoded into the output vector E of the probability of the user requesting a file in the file system at the next moment prod , whose length 5000 is the number of data objects available for pre-fetching in the file system. In this embodiment, the output vector E prod =[0.01, ..., 0.1, 0.44, ..., 0.352, ..., 0.01] represent the probability of the corresponding data object in the file system being requested at the next moment. The specific process is as follows Fig.16 shown.

[0122] S111, select output vector E prod In this embodiment, the top 5 high values ​​are {0.01, 0.1, 0.44, 0.352, 0.01}, which correspond to {d1, d 16 , d 17 , d 66 , d 5000}Data object. Read the above data object from the file system and store it in the cache.

[0123] Therefore, by converting the data access log into a graph form, the information retention is higher than the traditional sequence structure, which can better model the data access pattern and make it easier to learn complex global access patterns rather than just local access patterns. In addition, there is no need to manually construct features, and there is no need to set complex replacement and pre-fetching rules in advance. Instead, the complex data access patterns that are difficult to discover manually are automatically learned through graph representation learning, which reduces the dependence on artificial rules and eliminates the deviation caused by artificial induction. In addition, it supports parallel training and reasoning, improves the efficiency of calculations, can cluster training and reasoning, and has strong scalability. In addition, the data access log is modeled as a weighted directed non-self-loop graph, and the correlation of data access is learned using a graph neural network, and then the graph embedding is decoded by a decoder to predict the user's data access at the next moment, thereby reducing the number of cache misses and delays. This method has a high hit rate compared to the cache replacement and pre-fetching methods based on rules and traditional machine learning, and performs better in more complex data access patterns, with a wider range of promotion scenarios and service value. Among them, the service value can include improving the performance of the distributed cache system. This method effectively reduces the common overhead problems in the cache system and improves the cache hit rate; improving user experience, that is, it can accelerate user access to data, improve response speed and real-time performance, thereby improving platform user satisfaction and loyalty; reducing operation and maintenance costs. This method can reduce the capacity pressure on the cache area, reduce the resource consumption and failure risk of the cache area, thereby reducing operation and maintenance costs and complexity; increasing competitiveness, that is, it can improve the throughput and scalability of the distributed cache system, support more users and business scenarios, thereby increasing the competitiveness and market share of our products.

[0124] In summary, the embodiments of the present application can be based on a data processing method for cache prefetching of a graph neural network with a weighted directed graph without self-loops. Compared with the cache prefetching scheme in the related art, the data access log can be innovatively converted into a weighted directed graph without self-loops. Compared with the traditional sequence structure, the weighted directed graph without self-loops structure retains more diversified access pattern information of single users and multiple users; by modeling data access as a weighted directed graph, the access footprints of multiple users and the access patterns of multiple users can be aggregated to explore the diversity of user access patterns in multi-user scenarios; by prefetching through graph neural network data objects, complex user request patterns can be learned, while considering the access pattern information of the current user and the access pattern information of other users, and exploring request patterns from both single-user and multi-user perspectives, thereby improving the cache hit rate and reducing data access latency.

[0125] The present application also provides a data processing device, specifically in combination with Fig.17 Provide detailed explanation.

[0126] Fig.17It is a structural diagram of a data processing device provided by an embodiment of the present application.

[0127] In some embodiments of the present application, Fig.17 The data processing device shown can be set in the electronic device provided in the embodiment of the present application.

[0128] like Fig.17 As shown, the data processing device 170 may specifically include:

[0129] The acquisition module 1701 is used to acquire data access logs of N users accessing data objects, where N is an integer greater than 1;

[0130] A construction module 1702 is used to construct a weighted directed graph without self-loops of multi-user data access based on the data access log, wherein the weighted directed graph without self-loops includes M first nodes and a first directed connection edge connecting every two first nodes among the M first nodes, wherein the first node is the i-th accessed data object of at least one user among the N users, and the first directed connection edge is used to indicate the importance of the order of at least one user requesting to access the data object from one first node connected to the first directed connection edge to another first node in the data access log, i∈[1,M], where M is an integer greater than 1;

[0131] Extraction module 1703, used to extract P sampling subgraphs from the weighted directed self-loop-free graph, the sampling subgraphs including the second node and the second directed connection edge connecting the path where the second node is located, the second node is a node whose node out-degree is greater than or equal to the preset node out-degree among the M first nodes, and P is an integer greater than 1;

[0132] Training module 1704 is used to train the first graph neural network and pre-fetch decoder model based on each sampling subgraph in P sampling subgraphs and the label vector corresponding to each sampling subgraph until the preset training conditions are met to obtain the second graph neural network and pre-fetch decoder, and the second graph neural network and pre-fetch decoder are used to generate a probability value of the target user accessing the data object with the associated relationship in the second time period based on the data access log of the target user in the first time period.

[0133] In this way, the data access log can be modeled as a weighted directed graph without self-loops, effectively retaining the data access pattern of a single user, such as access order, access frequency, etc., effectively modeling a longer access sequence, reducing redundant information, and using graph neural networks and prefetch decoder models to learn the correlation of access data. It can effectively model the access patterns of multiple users, long sequences, and multiple sessions, adapt to the dynamic changes of data, and adapt to the data access scenarios of multiple users. While retaining the access data of a single user in the multi-user access scenario, the graph neural network and prefetch decoder model can capture more complex access data than traditional sequence data, encode more information containing Embedded vectors, learn access pattern representations in multi-user scenarios, and thus more accurately predict the user's data access in the next time period, reduce the number of cache misses and delays, improve the cache prefetch hit rate, and reduce frequent access to databases or disk files, thereby improving system response speed and performance.

[0134] The data processing device 170 in the embodiments of the present application is described in detail below.

[0135] In some embodiments of the present application, the construction module 1702 may be specifically used to construct a data access log sequence set based on the data access log, where the data access log sequence set includes a data access log sequence of each user among N users accessing a data object, and the data access log sequence includes an ordered set of data objects requested to be accessed by the user in the same session, where N is an integer greater than 1, i∈[1, N];

[0136] Based on the data access log sequence of each user among N users accessing the data object, a weighted directed self-loop-free graph of multi-user data access is constructed.

[0137] In some embodiments of the present application, the construction module 1702 may be specifically used to extract, from the data access log, a data access log sequence of each user among N users in units of sessions;

[0138] The data access log sequence of each user among the N users is aggregated to obtain a data access log sequence set.

[0139] In some embodiments of the present application, the construction module 1702 can be specifically used to generate a first weighted directed graph without self-loops of the jth user based on a data access log sequence of the jth user among N users accessing a data object, wherein the first weighted directed graph without self-loops includes K third nodes and a third directed connection edge connecting every two third nodes among the K third nodes, the K third nodes are data objects requested to be accessed by the jth user in the same session, and the third directed connection edge is used to indicate the importance of the order of the jth user from one third node connected by the third directed connection edge to another third node in the data access log sequence of the jth user accessing the data object to the order of the jth user accessing the K third nodes;

[0140] The second weighted directed self-loop-free graph of the users except the jth user among the N users is merged into the first weighted directed self-loop-free graph to obtain a weighted directed self-loop-free graph for multi-user data access

[0141] In some embodiments of the present application, the extraction module 1703 can be specifically used to, in a weighted directed graph without self-loops, correspond to a node out-degree for each first node, and the node out-degree is used to characterize the number of edges of the first node with the first node as the tail in the weighted directed graph without self-loops; in the case where P sampling subgraphs include a sampling subgraph of each second node in P second nodes, randomly select P second nodes whose node out-degree is greater than or equal to the preset node out-degree from the M first nodes in the weighted directed graph without self-loops;

[0142] Based on each second node among the P second nodes, extract the previous second node that is R hops before each second node, the subsequent second node after each second node, and the second directed connection edge that connects each second node, the previous second node, and the subsequent second node in series;

[0143] A sampling subgraph of each second node is generated according to each second node, the second node before each second node R hops ago, the successor second node of each second node, and the second directed connection edge connecting each second node, the second node before each second node, and the successor second node in series.

[0144] In some embodiments of the present application, the data processing device 170 in the embodiment of the present application may further include a partitioning module and a calculation module; wherein,

[0145] A partitioning module is used to partition a Q-th sampling subgraph among the P sampling subgraphs, in the case where the P sampling subgraphs include each sampling subgraph among the P sampling subgraphs and a label vector corresponding to each sampling subgraph, to obtain an input subgraph and an input label vector of the Q-th sampling subgraph, wherein the sampling subgraph includes a target node in the sampling subgraph, a target previous hop node T hops before the target node, and a first target directed connection edge connecting the target node and the target previous hop node in series, and the input label vector is a normalized vector of a second target directed connection edge connecting the target node and the target successor node of the target node in series, T∈[1,R], Q∈[1,P];

[0146] A calculation module is used to calculate the label vector of the Qth sampling subgraph according to the target node, the target successor node, the source node and the terminal node in the Qth sampling subgraph, and the first directed connection edge from the source node to the terminal node in the weighted directed acyclic graph. The label vector is used to characterize the probability that the node corresponding to the data object in the data access log is the target successor node after the target node.

[0147] In some embodiments of the present application, the computing module may be specifically used to, when the first graph neural network and the pre-fetch decoder model include a first graph neural network model and a first pre-fetch decoder model; encode each sampling subgraph through the first graph neural network model to obtain a first representation vector for representing the data access pattern;

[0148] Decoding the first representation vector by a first pre-fetch decoder model to obtain a first probability value of a user requesting to access the data object in a third time period;

[0149] When the cross entropy loss value between the vector of the first probability values ​​and the label vector is less than or equal to the preset cross entropy loss value, it is determined that the preset training condition is met, and a second graph neural network and a pre-fetch decoder are obtained.

[0150] In some embodiments of the present application, the data processing device 170 in the embodiment of the present application may also include a generating module, a processing module and a screening module; wherein,

[0151] The acquisition module 1701 may also be used to acquire a target data access log sequence of a target user accessing a data object within a first time period;

[0152] A generating module, used for generating a second weighted directed non-self-loop graph of a target user based on a target data access log sequence;

[0153] A processing module is used to perform graph enhancement processing on the second weighted directed graph without self-loops of the target user through the weighted directed graph without self-loops of multi-user data access to obtain a third weighted directed graph without self-loops; and, by processing the third weighted directed graph without self-loops through a second graph neural network and a pre-fetch decoder, obtain a probability value of the target user accessing a data object associated with the target data access log sequence in a second time period;

[0154] The screening module is used to screen the target data objects and store them in the cache based on the probability value of the target user accessing the data objects associated with the target data access log sequence in the second time period, and the probability value of the target data objects is greater than or equal to the preset probability value.

[0155] In some embodiments of the present application, the generating module may be specifically configured to arrange the data objects accessed by the target user in the first time period in the target data access log sequence according to the order in which the target user accesses the data objects in the first time period to obtain a data object set;

[0156] Based on the number of times the target user accesses the second target data object after accessing the first target data object in the data object set, generating a fourth directed connection edge connecting the first access node corresponding to the first target data object and the second access node corresponding to the second target data in series;

[0157] The first target data object and the second target data object are connected in series through the fourth directed connection edge to generate a second weighted directed graph without self-loops of the target user.

[0158] In some embodiments of the present application, the processing module may be specifically used to filter out a target terminal node in the second weighted directed self-loop-free graph from the weighted directed self-loop-free graph of multi-user data access;

[0159] Extract the target previous hop node D hops before the target terminal node from the weighted directed self-loop-free graph of multi-user data access, and obtain an enhanced subgraph;

[0160] The enhanced subgraph and the second weighted directed graph without self-loops are merged to obtain a third weighted directed graph without self-loops.

[0161] Optional, such as Fig.18 As shown, an embodiment of the present application further provides an electronic device 180, including a processor 1801 and a memory 1802, wherein the memory 1802 stores programs or instructions that can be executed on the processor 1801, and when the program or instructions are executed by the processor 1801, the various steps of the above-mentioned data processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0162] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0163] Fig.19 It is a schematic diagram of the structure of a computer device provided by an embodiment of the present application.

[0164] like Fig.19 As shown, the computer device may include at least one of the following involved in the embodiments of the present application: an electronic device, a server. The computer device may include a processor 1901 and a memory 1902 storing computer program instructions.

[0165] Specifically, the processor 1901 may include a central processing unit (CPU), or an application specific integrated circuit (ASTC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0166] The memory 1902 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1902 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 1902 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1902 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1902 is a non-volatile solid-state memory. In a specific embodiment, the memory 1902 includes a solid-state storage (ROM). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0167] The processor 1901 implements any one of the data processing methods in the above embodiments by reading and executing computer program instructions stored in the memory 1902 .

[0168] In one example, the computer device may further include a communication interface 1903 and a bus 1910. Fig.19 As shown, the processor 1901, the memory 1902, and the communication interface 1903 are connected via a bus 1910 and communicate with each other.

[0169] The communication interface 1903 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0170] Bus 1910 includes hardware, software or both, coupling the components of the flow control device to each other. For example and not limitation, the bus may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard system (ETSA) bus, a front-side bus (FSB), a hypertransport (HT) interconnect, an industry standard system (TSA) bus, an infinite bandwidth interconnect, a low pin count (LPC) bus, a memory bus, a microchannel system (MCA) bus, a peripheral component interconnect (PCT) bus, a PCT-Express (PCT-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus or other suitable bus or a combination of two or more of these. Where appropriate, bus 1910 may include one or more buses. Although the present application embodiment describes and illustrates a specific bus, the present application considers any suitable bus or interconnect.

[0171] The computer device can execute the data processing method in the embodiment of the present application, thereby realizing the combination Figures 1 to 19 The data processing method and device described.

[0172] In addition, in combination with the data processing method in the above embodiments, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiments is implemented.

[0173] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0174] The functional blocks shown in the above structural block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASTC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" may include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0175] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0176] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A data processing method, characterized in that: include: Obtain data access logs of N users accessing data objects, where N is an integer greater than 1; Based on the data access log, a weighted directed graph without self-loops of multi-user data access is constructed, wherein the weighted directed graph without self-loops comprises M first nodes and a first directed connection edge connecting every two first nodes of the M first nodes, wherein the first node is the i-th accessed data object of at least one user among the N users, and the first directed connection edge is used to indicate the importance of the order of the at least one user's request to access the data object from one first node connected to the first directed connection edge to another first node in the order of the N users accessing the M first nodes in the data access log, where M is an integer greater than 1, i∈[1,M]; Extracting P sampling subgraphs from the weighted directed self-loop-free graph, wherein the sampling subgraphs include a second node and a second directed connection edge that concatenates a path where the second node is located, wherein the second node is a node whose node out-degree is greater than or equal to a preset node out-degree among the M first nodes, and P is an integer greater than 1; Based on each sampling subgraph in the P sampling subgraphs and the label vector corresponding to each sampling subgraph, the first graph neural network and pre-fetch decoder model are trained until preset training conditions are met to obtain a second graph neural network and pre-fetch decoder, wherein the second graph neural network and pre-fetch decoder are used to generate a probability value of the target user accessing a data object with an associated relationship in a second time period based on the data access log of the target user in the first time period.

2. The data processing method according to claim 1, characterized in that: The step of constructing a weighted directed graph without self-loops of multi-user data access based on the data access log includes: Based on the data access log, construct a data access log sequence set, wherein the data access log sequence set includes a data access log sequence of each user among the N users accessing a data object, and the data access log sequence includes an ordered set of data objects requested to be accessed by the user in the same session, where N is an integer greater than 1, i∈[1,N]; Based on the data access log sequence of each user among the N users accessing the data object, a weighted directed graph without self-loops of the multi-user data access is constructed.

3. The data processing method according to claim 2, characterized in that: The step of constructing a data access log sequence set based on the data access log includes: Extracting the data access log sequence of each user among the N users in units of sessions from the data access log; The data access log sequence of each user among the N users is aggregated to obtain a data access log sequence set.

4. The data processing method according to claim 2 or 3, characterized in that: The step of constructing a weighted directed graph without self-loops of multi-user data access based on a data access log sequence of each user among the N users accessing the data object comprises: Based on a data access log sequence of a j-th user among the N users accessing a data object, generating a first weighted directed graph without self-loops of the j-th user, wherein the first weighted directed graph without self-loops comprises K third nodes and a third directed connection edge connecting every two third nodes among the K third nodes, the K third nodes are data objects requested to be accessed by the j-th user in the same session, and the third directed connection edge is used to indicate the importance of the order of the j-th user from one third node connected by the third directed connection edge to another third node in the data access log sequence of the j-th user accessing the data object to the order of the j-th user accessing the K third nodes; The second weighted directed graphs without self-loops of the users among the N users except the j-th user are merged into the first weighted directed graph without self-loops to obtain the weighted directed graph without self-loops of the multi-user data access.

5. The data processing method according to claim 1, characterized in that: Each first node in the weighted directed graph without self-loops corresponds to a node out-degree, and the node out-degree is used to represent the number of edges of the first node in the weighted directed graph without self-loops with the first node as the tail; The P sampling subgraphs include a sampling subgraph of each second node in the P second nodes; The step of extracting P sampling subgraphs from the weighted directed self-loop-free graph comprises: Randomly select P second nodes whose node out-degree is greater than or equal to a preset node out-degree from the M first nodes in the weighted directed graph without self-loops; Based on each second node among the P second nodes, extract the previous second node that is R hops before each second node, the subsequent second node after each second node, and the second directed connection edge that connects each second node, the previous second node, and the subsequent second node in series; A sampling subgraph of each second node is generated according to each second node, the second node that is R hops before each second node, the successor second node of each second node, and the second directed connection edge that connects each second node, the second node that is R hops before each second node, and the successor second node.

6. The data processing method according to claim 1, characterized in that: The P sampling subgraphs include each sampling subgraph in the P sampling subgraphs and a label vector corresponding to each sampling subgraph; the data processing method further includes: Divide the Qth sampling subgraph among the P sampling subgraphs to obtain an input subgraph and an input label vector of the Qth sampling subgraph, wherein the sampling subgraph includes a target node in the sampling subgraph, a target previous hop node T hops before the target node, and a first target directed connection edge connecting the target node and the target previous hop node in series, and the input label vector is a normalized vector of a second target directed connection edge connecting the target node and a target successor node of the target node in series, T∈[1,R],Q∈[1,P]; Based on the target node, the target successor node, the source node and the terminal node in the Qth sampling subgraph, and the first directed connection edge from the source node to the terminal node in the weighted directed acyclic graph, the label vector of the Qth sampling subgraph is calculated, and the label vector is used to characterize the probability that the node corresponding to the data object in the data access log is the target successor node after the target node.

7. The data processing method according to claim 1, characterized in that: The first graph neural network and pre-fetch decoder model include a first graph neural network model and a first pre-fetch decoder model; The first graph neural network and the pre-fetch decoder model are trained based on each sampling sub-graph in the P sampling sub-graphs and the label vector corresponding to each sampling sub-graph until a preset training condition is met to obtain a second graph neural network and a pre-fetch decoder, including: Encoding each of the sampling subgraphs through the first graph neural network model to obtain a first representation vector for representing a data access pattern; Decoding the first representation vector by using the first pre-fetch decoder model to obtain a first probability value of a user requesting to access the data object in a third time period; When the cross entropy loss value between the vector of the first probability values ​​and the label vector is less than or equal to a preset cross entropy loss value, it is determined that the preset training condition is met, and the second graph neural network and pre-fetch decoder are obtained.

8. The data processing method according to claim 1, characterized in that: The data processing method further includes: Acquire a target data access log sequence of the target user accessing the data object within a first time period; Based on the target data access log sequence, generating a second weighted directed non-self-loop graph of the target user; Performing graph enhancement processing on the second weighted directed graph without self-loops of the target user through the weighted directed graph without self-loops of the multi-user data access to obtain a third weighted directed graph without self-loops; Processing the third weighted directed non-self-loop graph by the second graph neural network and the pre-fetch decoder to obtain a probability value of the target user accessing a data object associated with the target data access log sequence during the second time period; Based on the probability value of the target user accessing the data object associated with the target data access log sequence in the second time period, the target data object is screened and stored in the cache, and the probability value of the target data object is greater than or equal to a preset probability value.

9. The data processing method according to claim 8, characterized in that: The step of generating a second weighted directed non-self-loop graph of the target user based on the target data access log sequence includes: Arranging the data objects accessed by the target user in the first time period in the target data access log sequence according to the order in which the target user accesses the data objects in the first time period to obtain a data object set; Based on the number of times the target user accesses the second target data object after accessing the first target data object in the data object set, generating a fourth directed connection edge that connects in series the first access node corresponding to the first target data object and the second access node corresponding to the second target data object; The first target data object and the second target data object are connected in series through the fourth directed connection edge to generate a second weighted directed graph without self-loops of the target user.

10. The data processing method according to claim 8, characterized in that: The weighted directed graph without self-loops accessed through the multi-user data performs graph enhancement processing on the second weighted directed graph without self-loops of the target user to obtain a third weighted directed graph without self-loops, including: Filtering out a target terminal node in the second weighted directed self-loop-free graph from the weighted directed self-loop-free graph of multi-user data access; Extracting a target previous hop node that is D hops before the target terminal node from the weighted directed non-self-loop graph of multi-user data access to obtain an enhanced subgraph; The enhanced subgraph and the second weighted directed graph without self-loops are merged to obtain the third weighted directed graph without self-loops.

11. A data processing device, characterized in that: include: An acquisition module is used to acquire data access logs of N users accessing data objects, where N is an integer greater than 1; A construction module, used for constructing a weighted directed graph without self-loops of multi-user data access based on the data access log, wherein the weighted directed graph without self-loops comprises M first nodes and a first directed connection edge connecting every two first nodes of the M first nodes, wherein the first node is the i-th accessed data object of at least one user among the N users, and the first directed connection edge is used to indicate the importance of the order of the at least one user's request to access the data object from one first node connected to the first directed connection edge to another first node in the order of the N users accessing the M first nodes in the data access log, i∈[1,M], where M is an integer greater than 1; An extraction module is used to extract P sampling subgraphs from the weighted directed self-loop-free graph, wherein the sampling subgraphs include a second node and a second directed connection edge that concatenates the path where the second node is located, the second node is a node whose node out-degree is greater than or equal to a preset node out-degree among the M first nodes, and P is an integer greater than 1; A training module is used to train a first graph neural network and a pre-fetch decoder model based on each sampling subgraph in the P sampling subgraphs and a label vector corresponding to each sampling subgraph until a preset training condition is met to obtain a second graph neural network and a pre-fetch decoder, wherein the second graph neural network and the pre-fetch decoder are used to generate a probability value of the target user accessing a data object with an associated relationship in a second time period based on the data access log of the target user in the first time period.

12. A computer device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the data processing method according to any one of claims 1 to 10 is implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the data processing method according to any one of claims 1 to 10 are implemented.

Images