Data caching method, device and equipment

Abstract

The invention relates to the technical field of computer data processing, provides a data caching method, device and equipment, and is used for solving the problem of long delay of data access of a user caused by low request success rate of data access of the user in related technologies. According to the data caching method provided by the embodiment of the invention, the frequency that each user requests the target data cached by the plurality of caching nodes in the first period can be determined based on the user request information of the data cached by the caching nodes in the previous period; and under the condition that the cache space occupied by the data cached by the cache nodes is not greater than the space capacity of the cache nodes, the storage states of the data in different cache nodes are adjusted, the average success rate of the data requested by the user is maximized, and the placement strategy of the data is optimized, so that each piece of data is placed in a proper cache node, and the user experience is improved. User access is facilitated, the cache hit rate is increased, and the delay of data request by the user is reduced.

Description

Technical Field

[0001] This application relates to the technical field of computer data processing, and particularly relates to a data caching method, apparatus, and device. Background Art

[0002] With the rapid development of the mobile Internet, the data traffic of data caching nodes has increased sharply, posing challenges to content transmission. Edge caching stores data content in edge caching nodes closer to users to reduce the latency of user requests for data content. This edge caching method is based on the "similarity" of user data requirements, predicts data requests, and pre-gets data from the central server to the edge caching nodes to cope with the increasing traffic.

[0003] In related technologies, although increasing the number of caching nodes can improve the downlink capacity of caching nodes to send data to users, due to the large number of caching nodes and the increase in data traffic, it is difficult for the central server to send data to the optimal caching nodes, and users cannot obtain data from the corresponding caching nodes, resulting in a low success rate of user requests to access data and a long latency for users to access data. Summary of the Invention

[0004] The purpose of this application is to provide a data caching method, apparatus, and device to solve the problem in related technologies that the success rate of user requests to access data is low, resulting in a long latency for users to access data.

[0005] In a first aspect, this application provides a data caching method, and the method includes:

[0006] Based on the user request information of multiple data cached by multiple caching nodes within a first period, determine the frequency of each user requesting the target data cached by the multiple caching nodes within the first period;

[0007] Based on the frequency of each user requesting the target data cached by the multiple caching nodes within the first period, determine the storage status of the target data in each caching node within a second period when the cache space occupied by the data cached in each caching node satisfies a first constraint condition and the average success rate of user requests for data satisfies a second constraint condition; where the second period is after the first period and is adjacent to the first period;

[0008] Based on the storage status of the target data in each caching node within the second period, cache the target data to the corresponding caching node.

[0009] In a possible implementation, the first constraint condition is that the cache space occupied by the data cached in each cache node is not greater than the cache space capacity of each cache node, and the second constraint condition is to maximize the average success rate of user requests for data.

[0010] In a possible implementation, determining the frequency at which each user requests the target data cached in the multiple cache nodes during the first period based on the user request information of the multiple data cached in the multiple cache nodes during the first period includes:

[0011] For any one user, based on the user request information of the multiple data cached in the multiple cache nodes during the first period, determining the number of times the any one user requests the target data; and the sum of the number of times the any one user requests multiple target data;

[0012] Based on the number of times the any one user requests the target data and the sum of the number of times the any one user requests multiple target data, determining the frequency at which the any one user requests the target data cached in the multiple cache nodes during the first period.

[0013] In a possible implementation, determining the storage state of the target data in each cache node during the second period when the cache space occupied by the data cached in each cache node satisfies the first constraint condition and the average success rate of user requests for data satisfies the second constraint condition based on the frequency at which each user requests the target data cached in the multiple cache nodes during the first period includes:

[0014] Based on the number of cache nodes, combining the storage states of the target data corresponding to each cache node to obtain multiple cache state sets, and summarizing the multiple cache state sets corresponding to the multiple target data to obtain a total cache state set, where each cache state set includes a storage state corresponding one-to-one to the cache nodes, and each storage state indicates whether the target data is cached in the corresponding cache node;

[0015] Traversing each cache state set in the total cache state set and performing the following process:

[0016] Based on the storage states in each traversed cache state set and the frequency at which each user requests the target data cached in the multiple cache nodes during the first period, obtaining the average success rate of user requests for data; and based on the storage states in each traversed cache state set, obtaining the cache space occupied by the multiple target data cached in each cache node;

[0017] Determine from the sets of cache states where the cache spaces occupied by the multiple pieces of target data cached in each cache node are not greater than the cache capacity of each cache node, and the set of cache states with the highest average success rate of the corresponding user-requested data;

[0018] Use the storage state in the selected set of cache states as the storage state of the target data in each cache node during the second period.

[0019] In a possible implementation manner, obtaining the average success rate of user-requested data based on the storage states in the traversed sets of cache states and the frequencies of the multiple pieces of target data cached in the multiple cache nodes by each user during the first period includes:

[0020] For any user, determine the target cache node corresponding to the any user and the adjacent nodes of the target cache node, and based on the storage state of the target data corresponding to the target cache node and the storage state of the target data corresponding to the adjacent nodes of the target cache node, determine the access state of the target data; the access state indicates whether the target data can be accessed by the any user;

[0021] Based on the access state of the target data and the frequency of the any user requesting the target data during the first period, determine the request success rate of the any user for the target data;

[0022] Sum the multiple request success rates of the any user for multiple pieces of the target data to obtain the request success rate of the any user for the requested data;

[0023] Average the sum of the request success rates of the multiple any-user-requested data corresponding to each user to obtain the average success rate of user-requested data.

[0024] In a possible implementation manner, determining the access state of the target data based on the storage state of the target data corresponding to the target cache node and the storage state of the target data corresponding to the adjacent nodes of the target cache node includes:

[0025] Based on the storage state of the target data corresponding to the target cache node and the storage state of the target data corresponding to the adjacent nodes of the target cache node, if it is determined that the target data is cached in any one of the cache nodes within a preset range, determine that the target data can be accessed by the any user; the cache nodes within the preset range are the target cache node corresponding to the any user and the adjacent nodes of the target cache node;

[0026] If it is determined that the target data is not cached in any of the cache nodes within a preset range, it is determined that the target data is not accessed by any of the users.

[0027] In a possible implementation manner, after determining the storage status of the target data in each cache node during the second period and before caching the target data into the corresponding cache node, it further includes:

[0028] Clear the data cached in each cache node, and cache the data of the same type as the target data in the newly added data into the cache node corresponding to the target data; the newly added data is the data received by the central server during the first period.

[0029] In a second aspect, the present application provides a data caching device, and the device includes:

[0030] A frequency determination module, configured to determine the frequency of each user requesting the target data cached in the multiple cache nodes during the first period based on the user request information of the multiple data cached in the multiple cache nodes during the first period;

[0031] A data prediction module, configured to determine the storage status of the target data in each cache node during the second period when the cache space occupied by the data cached in each cache node satisfies a first constraint condition and the average success rate of user-requested data satisfies a second constraint condition based on the frequency of each user requesting the target data cached in the multiple cache nodes during the first period; where the second period is after the first period and is adjacent to the first period;

[0032] A data sending module, configured to cache the target data into the corresponding cache node based on the storage status of the target data in each cache node during the second period.

[0033] In a possible implementation manner, the first constraint condition is that the cache space occupied by the data cached in each cache node is not greater than the cache space capacity of each target cache node, and the second constraint condition is to maximize the average success rate of user-requested data.

[0034] In a possible implementation manner, for determining the frequency of each user requesting the target data cached in the multiple cache nodes during the first period based on the user request information of the multiple data cached in the multiple cache nodes during the first period, the frequency determination module is specifically configured to:

[0035] For any user, determine the number of times the any user requests the target data and the sum of the number of times the any user requests multiple target data based on the user request information of the multiple data cached in the multiple cache nodes during the first period;

[0036] Based on the sum of the number of times any one user requests the target data and the number of times any one user requests multiple target data, determine the frequency at which any one user requests the target data cached by the multiple cache nodes within the first period.

[0037] In a possible implementation manner, for the data prediction module to determine the storage status of the target data in each cache node within the second period when the cache space occupied by the data cached in each cache node meets the first constraint condition and the average success rate of user data requests meets the second constraint condition, based on the frequency at which each user requests the target data cached by the multiple cache nodes within the first period, the data prediction module specifically is used for:

[0038] Based on the number of cache nodes, combine the storage statuses of the target data corresponding to each cache node to obtain multiple cache status sets, and summarize the multiple cache status sets corresponding to the multiple target data to obtain a total cache status set, where each cache status set includes a storage status corresponding one-to-one to the cache nodes, and each storage status represents whether the target data is cached to the corresponding cache node;

[0039] Traverse each cache status set in the total cache status set and perform the following process:

[0040] Based on the storage statuses in each traversed cache status set and the frequency at which each user requests the target data cached by the multiple cache nodes within the first period, obtain the average success rate of user data requests; and based on the storage statuses in each traversed cache status set, obtain the cache space occupied by the multiple target data cached in each cache node;

[0041] Determine from the cache status sets that satisfy that the cache space occupied by the multiple target data cached in each cache node is not greater than the cache capacity of each cache node and whose corresponding average success rate of user data requests is the largest;

[0042] Use the storage statuses in the selected cache status set as the storage statuses of the target data in each cache node within the second period.

[0043] In a possible implementation manner, for the data prediction module to obtain the average success rate of user data requests based on the storage statuses in each traversed cache status set and the frequency at which each user requests the target data cached by the multiple cache nodes within the first period, the data prediction module specifically is used for:

[0044] For any user, determine the target cache node corresponding to the any user and the adjacent nodes of the target cache node, and determine the access status of the target data based on the storage status of the target data corresponding to the target cache node and the storage status of the target data corresponding to the adjacent nodes of the target cache node; the access status indicates whether the target data can be accessed by the any user;

[0045] Based on the access status of the target data and the frequency of the any user requesting the target data within the first period, determine the request success rate of the any user for the target data;

[0046] Sum up the multiple request success rates of the any user for multiple target data to obtain the request success rate of the any user for requesting data;

[0047] Average the sum of the request success rates of multiple any user for requesting data corresponding to each user to obtain the average success rate of users for requesting data.

[0048] In a possible implementation manner, for the determining the access status of the target data based on the storage status of the target data corresponding to the target cache node and the storage status of the target data corresponding to the adjacent nodes of the target cache node, the data prediction module specifically is used for:

[0049] Based on the storage status of the target data corresponding to the target cache node and the storage status of the target data corresponding to the adjacent nodes of the target cache node, if it is determined that the target data is cached in any one of the cache nodes within a preset range of cache nodes, then determine that the target data can be accessed by the any user; the cache nodes within the preset range are the target cache node corresponding to the any user and the adjacent nodes of the target cache node;

[0050] If it is determined that the target data is not cached in any one of the cache nodes within the preset range of cache nodes, then determine that the target data is not accessed by the any user.

[0051] In a possible implementation manner, after determining the storage status of the target data in each cache node within the second period and before caching the target data into the corresponding cache node, there is also a data clearing module, which is used for:

[0052] Clear the data cached in each cache node, and cache the data of the same type as the target data in the new added data into the cache node corresponding to the target data; the new added data is the data received by the central server within the first period.

[0053] In a third aspect, the present application provides an electronic device, including:

[0054] a processor and a memory;

[0055] The memory is used to store executable instructions of the processor;

[0056] The processor is configured to execute the instructions to implement the data caching method provided in any one of the first aspects of the present application.

[0057] In a fourth aspect, the present application provides a computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the data caching method provided in any one of the first aspects of the present application.

[0058] In a fifth aspect, the present application provides a computer program product, including a computer program, which when executed by a processor implements the data caching method provided in any one of the first aspects of the present application.

[0059] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0060] The data caching method provided by the embodiments of the present application, based on the user request information of the data cached by the cache node in the previous cycle, can, when the cache space occupied by the data cached by the cache node is not greater than the space capacity of the cache node, maximize the average success rate of the user-requested data by adjusting the storage status of the data in different cache nodes, optimize the data placement strategy, so that each data is placed in a suitable cache node, facilitating user access, improving the cache hit rate, reducing the latency of the user-requested data, and the technical solution provided by the present application has a simple method and good generality.

[0061] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. On the basis of conforming to the common knowledge in the art, the above preferred conditions can be combined arbitrarily to obtain various preferred embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0063] Figure 1 It is a schematic diagram of an application scenario of a data caching method provided by an embodiment of the present application;

[0064] Figure 2 It is a schematic diagram of the overall process of a data caching method provided by an embodiment of this application;

[0065] Figure 3 It is a schematic diagram of the process for determining the frequency of target data cached by multiple cache nodes for each user request within the first period provided by an embodiment of this application;

[0066] Figure 4 It is a schematic diagram of the process for determining the storage status of target data in each cache node within the second period provided by an embodiment of this application;

[0067] Figure 5 It is a schematic diagram of the process for determining the access status of target data provided by an embodiment of this application;

[0068] Figure 6 It is a schematic diagram of the overall process framework for user data requests provided by an embodiment of this application;

[0069] Figure 7 It is a schematic diagram of the structure of a data caching device provided by an embodiment of this application;

[0070] Figure 8 It is a schematic diagram of the structure of an electronic device provided by an embodiment of this application. Detailed implementation manners

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Among them, the described embodiments are some but not all of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of this application without creative efforts fall within the scope of protection of this application.

[0072] Moreover, in the description of the embodiments of this application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B; "and / or" in the text is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "a plurality of" means two or more than two.

[0073] Hereinafter, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0074] The following is an explanation of the professional terms and technologies involved in this application:

[0075] Data popularity: Data popularity (Content Popularity) refers to the frequency at which a certain data or content is accessed or requested within a certain period of time. It is usually used to describe and predict the degree of user demand for different contents. In fields such as content delivery networks (CDNs), caching systems, and information retrieval, data popularity is a key factor because it can help optimize content storage and transmission strategies to improve system efficiency and user experience.

[0076] Edge node: It refers to a node in a computer network that is located close to the data source or user terminal. It is an important part of the edge computing architecture and is responsible for processing and storing data to reduce data transmission latency and bandwidth consumption.

[0077] Caching policy algorithm: It refers to how to store data in the corresponding cache nodes. The caching policy algorithm is a set of rules or methods used to manage cache storage in a computer system or network, aiming to improve data access efficiency and system performance.

[0078] With the rapid development of the mobile Internet, the data traffic of data cache nodes has increased sharply, posing challenges to content transmission. Edge caching stores data content in edge cache nodes closer to users to reduce the latency of user requests for data content. This edge caching method is based on the "similarity" of user data needs, predicts data requests, and pre-empts data from the central server to the edge cache nodes to cope with the growing traffic.

[0079] In related technologies, although increasing the number of cache nodes can improve the downlink capacity of cache nodes to send data to users, due to the large number of cache nodes and the increase in data traffic, it is difficult for the central server to send data to the optimal cache nodes, and users cannot obtain data from the corresponding cache nodes, resulting in a low success rate of user requests to access data and a long latency for users to access data.

[0080] In view of this, this application provides a data caching method, device, and equipment to solve the problem in related technologies that the success rate of user requests to access data is low, resulting in a long latency for users to access data.

[0081] The inventive concept of the present application can be summarized as follows: First, based on the user request information of multiple data cached by multiple cache nodes within the first period, determine the frequency at which each user requests the target data cached by multiple cache nodes within the first period; then, based on the frequency at which each user requests the target data cached by multiple cache nodes within the first period, determine the storage state of the target data at each cache node within the second period when the cache space occupied by the data cached at each cache node meets the first constraint condition and the average success rate of user-requested data meets the second constraint condition; finally, based on the storage state of the target data at each cache node within the second period, cache the target data to the corresponding cache node.

[0082] In summary, the data caching method provided by the embodiments of the present application, based on the user request information of the data cached by the cache nodes in the previous period, can, when the cache space occupied by the data cached by the cache nodes is not greater than the space capacity of the cache nodes, maximize the average success rate of user-requested data by adjusting the storage state of the data at different cache nodes, optimize the data placement strategy, place each data at a suitable cache node, facilitate user access, improve the cache hit rate, reduce the latency of user-requested data, and the technical solution provided by the present application has a simple method and good generality.

[0083] After introducing the main inventive idea of the embodiments of the present application, the following briefly introduces the application scenarios applicable to the technical solutions of the embodiments of the present application. It should be noted that the following introduced application scenarios are only for illustrating the embodiments of the present application and not for limitation. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0084] For ease of understanding, the following details a data caching method provided by the embodiments of the present application with reference to the accompanying drawings:

[0085] As Figure 1 shown, it is a schematic diagram of the application scenario of a data caching method provided by the embodiments of the present application. The figure includes: network 10, server 20, and memory 30. The server 20 receives data through the network and, through the method provided by the embodiments of the present application, sends the data to the corresponding cache nodes, optimizing the data placement strategy, placing each data at a suitable cache node, facilitating user access, and improving the cache hit rate.

[0086] In the description of the present application, only a single server is described in detail. However, those skilled in the art should understand that the illustrated network 10, server 20, and memory 30 are intended to represent the operations of electronic devices, servers, and memories involved in the technical solution of the present application. The detailed description of a single server and memory is for convenience of illustration only, and does not imply any limitation on the number, type, or location of the servers. It should be noted that if additional modules are added to or individual modules are removed from the illustrated environment, the underlying concept of the exemplary embodiments of the present application will not be changed. Additionally, although bidirectional arrows from the memory 30 to the server 20 are shown in Figure 1 for convenience of illustration, those skilled in the art can understand that the above data transmission and reception also need to be implemented through the network 10.

[0087] It should be noted that the memory in the embodiments of the present application can be, for example, a cache system, a hard disk storage, a memory storage, etc. In addition, the data caching method proposed in the present application is not only applicable to Figure 1 the application scenarios shown, but can also be used in other possible application scenarios, and the embodiments of the present application do not impose any limitations.

[0088] Based on the above description, a data caching method provided by an embodiment of the present application has an overall process as Figure 2 shown and includes the following:

[0089] In step 201, based on the user request information of multiple data cached by multiple cache nodes within a first period, determine the frequency at which each user requests target data cached by multiple cache nodes within the first period.

[0090] In a possible implementation manner, in step 201, based on the user request information of multiple data cached by multiple cache nodes within a first period, determining the frequency at which each user requests target data cached by multiple cache nodes within the first period has a process as Figure 3 shown and can be implemented as:

[0091] In step 301, for any one user, based on the user request information of multiple data cached by multiple cache nodes within a first period, determine the number of times any one user requests target data; and the sum of the number of times any one user requests multiple target data.

[0092] In step 302, based on the number of times any one user requests target data and the sum of the number of times any one user requests multiple target data, determine the frequency at which any one user requests target data cached by multiple cache nodes within the first period.

[0093] For example, assume that there are K cache nodes and L users in the data distribution system. The sets of cache nodes and users are represented by N and U respectively, i.e., N = {n1, n2, …, n K , U = {u1, u2, …, u L}; Use Nr(u l ) to represent the set of cache nodes that user u l can use for data access. This set consists of the target cache nodes that u l can connect to and their adjacent nodes; Assume that the total number of data cached by K cache nodes is M, and the data forms a data set C = {c1, c2, …, c m , and Size(c i ) represents the size of data c i . Assume that the popularity of the user request information of multiple data cached by multiple cache nodes follows a Zipf distribution with parameter α (Zipf distribution), that is, the access frequency distribution of the data follows a Zipf distribution with parameter α, and the number of data accessed by the user per second is 1; Using the following formula (1), the embodiments of the present application can obtain the occurrence times of each data cached by the cache nodes:

[0094]

[0095] m represents the rank of the occurrence times of a data, P(m) represents the occurrence times of the data with rank m, the access frequency distribution of the data follows a Zipf distribution with parameter α, and B is a constant;

[0096] Based on the occurrence times of each data cached by the above cache nodes, using the following formula (2), the embodiments of the present application can determine the frequency of each data requested by different users:

[0097]

[0098] Among them, p lm represents the frequency of user u l requesting data c m , user u l represents the l-th user in the user set U, data c m represents the m-th data in the data set C, M represents the total number of data, represents the number of times user u l requests data c m , represents the number of times user u l requests the i-th data.

[0099] After the above formula (2) is deduced, the following formula (3) is obtained

[0100]

[0101] In step 202, based on the frequencies at which each user requests the target data cached in multiple cache nodes during the first period, determine the storage status of the target data in each cache node during the second period when the cache space occupied by the data cached in each cache node meets the first constraint condition and the average success rate of user requests for data meets the second constraint condition. It should be added that the second period is after the first period and is adjacent to the first period.

[0102] In a possible implementation manner, in step 202, based on the frequencies at which each user requests the target data cached in multiple cache nodes during the first period, determine the storage status of the target data in each cache node during the second period when the cache space occupied by the data cached in each cache node meets the first constraint condition and the average success rate of user requests for data meets the second constraint condition. The process is as Figure 4 shown and can be implemented as:

[0103] In step 401, based on the number of cache nodes, combine the storage statuses of the target data corresponding to each cache node to obtain multiple cache status sets, and summarize the multiple cache status sets corresponding to multiple target data to obtain the total cache status set.

[0104] It should be added that each cache status set includes storage statuses corresponding one-to-one to the cache nodes, and each storage status indicates whether the target data is cached in the corresponding cache node.

[0105] It should be noted that a cache status set is the storage status of a target data in each cache node. For each target data, when the storage status of any one cache node changes, a new permutation and combination of storage statuses is added. Therefore, a cache status set corresponds to a permutation and combination of the storage status of a target data in each cache node. Since each target data corresponds to multiple cache nodes and there are multiple storage statuses in each cache node, all the permutation and combination cases of the storage status corresponding to a target data are multiple cache status sets. For multiple target data, summarizing the multiple cache status sets corresponding to multiple target data gives the total cache status set.

[0106] For example, use I mk to represent the status of data c m in cache node n k Specifically:

[0107]

[0108] Then let x i = I mk, i = K(m - 1) + k, 1 ≤ m ≤ M, 1 ≤ k ≤ K, use the vector X = x1, x2, …, x MK represents the storage status of each piece of data in different cache nodes.

[0109] Assume the target data is c1, and the cache nodes include n1 and n2. For the current target data c1, there are 4 storage statuses in different cache nodes. I 11 = 1, I 12 = 1, I 11 = 0, I 12 = 0. Each cache status set includes the storage status corresponding one-to-one to the cache nodes, that is, the permutations and combinations of the above storage statuses include [I 11 = 1, I 12 = 1], [I 11 = 1, I 12 = 0], [I 11 = 0, I 12 = 1], [I 11 = 0, I 12 = 0] 4 kinds. One permutation and combination of the storage statuses in each bracket is a cache status set. Therefore, the target data c1 corresponds to 4 cache status sets. When there are 4 target data, summarize the cache status sets corresponding to the 4 target data respectively to obtain the total cache status set. For example, if each target data corresponds to 4 cache status sets, then the total cache status set includes 16 cache status sets.

[0110] In step 402, traverse each cache status set in the total cache status set and execute the following process:

[0111] Based on the storage status in each traversed cache status set and the frequency of each user requesting the target data cached in multiple cache nodes in the first period, obtain the average success rate of the user's requested data; and based on the storage status in each traversed cache status set, obtain the cache space occupied by the multiple target data cached in each cache node.

[0112] In a possible implementation manner, in step 402, based on the storage status in each traversed cache status set and the frequency of each user requesting the target data cached in multiple cache nodes in the first period, obtain the average success rate of the user's requested data. The process is as Figure 5 shown and can be implemented as:

[0113] In step 501, for any user, determine the target cache node corresponding to any user and the adjacent nodes of the target cache node. Based on the storage status of the target data in the target cache node and the storage status of the target data in the adjacent nodes of the target cache node, determine the access status of the target data. Here, the access status indicates whether the target data can be accessed by any user.

[0114] In a possible implementation manner, in step 501, based on the storage status of the target data in the target cache node and the storage status of the target data in the adjacent nodes of the target cache node, determining the access status of the target data includes the following two cases:

[0115] If it is determined that the target data is cached in any one of the cache nodes within a preset range, it is determined that the target data can be accessed by any user; the cache nodes within the preset range are the target cache node corresponding to any user and the adjacent nodes of the target cache node;

[0116] If it is determined that the target data is not cached in any one of the cache nodes within the preset range, it is determined that the target data cannot be accessed by any user.

[0117] For example, any user is u l , the target data is c1, and the cache nodes within the preset range include n1 and n2. If it is determined that the target data is cached in any one of the cache nodes within the preset range, that is, the target data c1 is cached in any one of n1 and n2, it indicates that the target data c1 can be accessed by the user u l and corresponds to the following 3 permutations and combinations of storage states [I 11 =1, I 12 =1], [I 11 =1, I 12 =0], [I 11 =0, I 12 =1]; if it is determined that the target data is not cached in any one of the cache nodes within the preset range, that is, the target data c1 is not cached in any one of n1 and n2, it indicates that the target data c1 cannot be accessed by the user u l and the corresponding permutation and combination of the storage state at this time is [I 11 =0, I 12 =0].

[0118] In step 502, based on the access status of the target data and the frequency of any user requesting the target data within the first period, determine the request success rate of any user requesting the target data.

[0119] In step 503, the success rates of multiple requests from any one user request to multiple target data are summed up to obtain the success rate of the data requested by any one user.

[0120] In step 504, the sum of the success rates of multiple data requested by any one user corresponding to each user is averaged to obtain the average success rate of the data requested by the user.

[0121] The average success rate f(X) of the data requested by the user is represented by the following formula (4):

[0122]

[0123] Among them, f(X) represents the average success rate of the data requested by the user, L represents the total number of users in the user set U, p lm represents the frequency of the user u l requesting the data c m , Nr(u l ) represents the set of cache nodes available for data access by the user u l (that is, the target cache node corresponding to the user u l and the adjacent nodes of the target cache node), x K(m-1)+j is the storage state of the cache node j for the data c m , is the access state of the data c m , which characterizes whether the data c m can be accessed by u l .

[0124] The above steps for determining the average success rate of the data requested by the user, taking into account the adjacent nodes of each cache node, can effectively determine the average success rate of the data requested by the user.

[0125] In step 403, it is determined from each cache state set that satisfies that the cache space occupied by multiple target data cached in each cache node is not greater than the cache capacity of each cache node, and the cache state set with the maximum average success rate of the corresponding user-requested data.

[0126] In step 404, the storage state in the selected cache state set is used as the storage state of the target data in each cache node in the second period.

[0127] Assume that the cache space capacity of the cache node n k is S k , then the cache status of the buffer node n k should satisfy the following inequality (5):

[0128]

[0129] Therefore, the set of cache states that maximizes the average success rate of user-requested data under the cache capacity constraint of the cache node is expressed as:

[0130] maxf(X)

[0131]

[0132] Finally, the storage state in the set of cache states that satisfies the above polynomial (6) is used as the storage state of the target data in each cache node in the second period.

[0133] Based on the user request information of the data cached in the cache node in the previous period and combined with the situation of the adjacent nodes of the cache node, the embodiments of the present application can, under the condition that the cache space occupied by the data cached in the cache node is not greater than the space capacity of the cache node, maximize the average success rate of user-requested data by adjusting the storage state of the data in different cache nodes, optimize the data placement strategy, place each data in a suitable cache node, facilitate user access, improve the cache hit rate, and reduce the latency of user-requested data.

[0134] In step 203, based on the storage state of the target data in each cache node in the second period, the target data is cached to the corresponding cache node.

[0135] In a possible implementation manner, after determining the storage state of the target data in each cache node in the second period and before caching the target data to the corresponding cache node, the embodiments of the present application also clear the data cached in each cache node and cache the data of the same type as the target data in the newly added data to the cache node corresponding to the target data, where the newly added data is the data received by the central server in the first period.

[0136] In another possible implementation manner, the problem of determining the set of cache states that maximizes the average success rate of user-requested data under the cache capacity constraint of the cache node (subsequently referred to as the cache policy problem) proposed in the above embodiments is an NP-Hard problem, and the proof is as follows:

[0137] Consider an instance with only one cache node and one user. Let the average success rate f * (x) in this instance be specifically:

[0138]

[0139] where p m represents the probability that the user accesses the data c m , represents whether the data c m is in the cache node;

[0140] In this instance, the cache policy problem is expressed as:

[0141] maxf * (X)

[0142]

[0143] The Knapsack problem is one of Karp's twenty-one NP-Complete problems, which can be described as follows: Given a set G of n items, each item in G has its own value p i and volume w i . Given a knapsack with a capacity of W, each item can only be selected 0 or 1 time. The problem is to find the combination of items in the knapsack that maximizes the total value. Specifically, in the embodiments of this application, the cache policy problem involved can be defined as:

[0144]

[0145]

[0146] Consider the following mapping:

[0147] Map the cache space capacity S of the cache node to the knapsack capacity W;

[0148] Map the data set C to the item set G;

[0149] Map the data size Size(c i ) to the item volume w i ;

[0150] Map the probability p m of the user accessing the data c m to the item value p i ,

[0151] Through this mapping, the cache policy problem of a single node and a single user can be transformed into a 0 / 1 knapsack problem. Assume that there is an algorithm A that can solve this instance of the cache policy problem in polynomial time. The optimal solution obtained by using algorithm A for this instance of the cache policy problem can definitely be transformed into the optimal solution of the 0 / 1 knapsack problem. Since the knapsack problem is an NP-Complete problem, the cache policy problem is an NP-Hard problem.

[0152] Since the original problem is an NP-Hard problem, the embodiments of this application simplify the original cache policy problem to solve for an approximate optimal solution. First, let:

[0153] Size(c i ) = 1, i = 1, 2,..., M

[0154] Then, add a new item that does not change its value to the cache policy problem. The cache policy problem then becomes:

[0155]

[0156] where β K(m-1)+k ≥ 0. Use y i ∈ [0, 1] to replace x i ∈ 0, 1. Then the cache policy problem is transformed into:

[0157]

[0158] where Y = y K(m-1)+k : 1 ≤ m ≤ M, 1 ≤ k ≤ K. Let -G(Y) be the sum of f(Y) and the added term. Then the diagonal elements of the Hessian matrix H(Y) of -G(Y) are {2β1, 2β2, …, 2β MK}. According to the Gerschgorin disk theorem, all eigenvalues of any square matrix A = [a ij are in the union of the Gerschgorin disks centered at the diagonal elements, that is Each disk contains at least one eigenvalue, and the radius of the disk is equal to the sum of the absolute values of the off-diagonal elements in each row of the square matrix A. The disk D ii centered at a i = {z ∈ C: |z - a ii | ≤ ∑ j≠i |a ij |}. So when β k satisfies the following formula (12), all eigenvalues of H(Y) are positive.

[0159]

[0160] Since the Hessian matrix is a symmetric matrix, so when β k satisfies formula (12), H(Y) is a positive definite matrix. At this time, G(Y) = -(f(Y) + ∑ m,k β K(m-1)+k y K(m-1)+k (1 - y K(m-1)+k )) is a convex function. So the cache policy problem is transformed into:

[0161] min G(Y)

[0162]

[0163] Suppose is a solution to the problem described by the formula. Since So it is necessary to use the stochastic rounding algorithm to round to Specifically, for each item in Y * this algorithm takes The probability is rounded to 1 to The probability is rounded to 0.

[0164] Therefore, the cache policy algorithm in the embodiments of this application can be expressed as the following algorithm:

[0165]

[0166]

[0167] In a possible implementation manner, based on the data caching method provided in the embodiments of this application, after the central server distributes data to each cache node, the overall process framework of user requests for data is as Figure 6 shown. The user of the client requests data A from the cache node. If there is a metadata cache of data A in the cache node or its neighboring node, the cache node returns data A to the client; if the user of the client requests data B from the cache node, and there is a metadata cache of data B in the cache node or its neighboring node, the cache node requests data B from the central server. After the central server returns data B to the cache node or its neighboring node, the cache node returns data B to the client.

[0168] In summary, the data caching method provided in the embodiments of this application, based on the user request information of the data cached by the cache node in the previous cycle, can, when the cache space occupied by the data cached by the cache node is not greater than the space capacity of the cache node, maximize the average success rate of user-requested data by adjusting the storage status of data in different cache nodes, optimize the data placement strategy, so that each piece of data is placed in a suitable cache node, facilitating user access, improving the cache hit rate, reducing the latency of user-requested data, and the technical solution provided in this application has a simple method and good generality.

[0169] Based on the same inventive concept, the embodiments of this application also provide a data caching device, as Figure 7 shown. The device 700 includes:

[0170] A frequency determination module 701, configured to determine the frequency of each user requesting the target data cached by the multiple cache nodes within the first cycle based on the user request information of the multiple data cached by the multiple cache nodes within the first cycle;

[0171] A data prediction module 702, configured to determine the storage status of the target data in each cache node in the second period when the cache space occupied by the data cached in each cache node meets the first constraint condition and the average success rate of user-requested data meets the second constraint condition, based on the frequencies of the target data requested by each user from the multiple cache nodes during the first period; wherein the second period is after the first period and adjacent to the first period.

[0172] A data sending module 703, configured to cache the target data into the corresponding cache node based on the storage status of the target data in each cache node during the second period.

[0173] In a possible implementation manner, the first constraint condition is that the cache space occupied by the data cached in each cache node is not greater than the cache space capacity of each cache node, and the second constraint condition is to maximize the average success rate of user-requested data.

[0174] In a possible implementation manner, for determining the frequencies of the target data requested by each user from the multiple cache nodes during the first period based on the user request information of the multiple data cached in the multiple cache nodes during the first period, the frequency determination module is specifically configured to:

[0175] For any user, determine the number of times the target data is requested by the user based on the user request information of the multiple data cached in the multiple cache nodes during the first period; and the sum of the number of times the user requests multiple target data;

[0176] Based on the number of times the target data is requested by the user and the sum of the number of times the user requests multiple target data, determine the frequency of the target data requested by the user from the multiple cache nodes during the first period.

[0177] In a possible implementation manner, for determining the storage status of the target data in each cache node in the second period when the cache space occupied by the data cached in each cache node meets the first constraint condition and the average success rate of user-requested data meets the second constraint condition, based on the frequencies of the target data requested by each user from the multiple cache nodes during the first period, the data prediction module is specifically configured to:

[0178] Based on the number of cache nodes, combine the storage states of the target data corresponding to each cache node to obtain multiple cache state sets, and summarize the multiple cache state sets corresponding to the multiple target data to obtain a total cache state set, where each cache state set includes storage states corresponding one-to-one to the cache nodes, and each storage state indicates whether the target data is cached in the corresponding cache node;

[0179] Traverse each cache state set in the total cache state set and perform the following process:

[0180] Based on the storage states in each traversed cache state set and the frequencies of each user requesting the target data cached in the multiple cache nodes within the first period, obtain the average success rate of user-requested data; and based on the storage states in each traversed cache state set, obtain the cache space occupied by the multiple target data cached in each cache node;

[0181] Determine from the cache state sets that satisfy that the cache space occupied by the multiple target data cached in each cache node is not greater than the cache capacity of each cache node, and the cache state set with the maximum average success rate of the corresponding user-requested data;

[0182] Use the storage states in the selected cache state set as the storage states of the target data in each cache node within the second period.

[0183] In a possible implementation manner, for obtaining the average success rate of user-requested data based on the storage states in each traversed cache state set and the frequencies of each user requesting the target data cached in the multiple cache nodes within the first period, the data prediction module is specifically configured to:

[0184] For any one user, determine the target cache node corresponding to the any one user and the adjacent nodes of the target cache node, and based on the storage state of the target data corresponding to the target cache node and the storage state of the target data corresponding to the adjacent nodes of the target cache node, determine the access state of the target data; the access state indicates whether the target data can be accessed by the any one user;

[0185] Based on the access state of the target data and the frequency of the any one user requesting the target data within the first period, determine the request success rate of the any one user requesting the target data;

[0186] Sum the multiple request success rates of the any one user requesting multiple target data to obtain the request success rate of the any one user requesting data;

[0187] Average the sum of the request success rates for multiple requests for any one of the user request data corresponding to each user to obtain the average success rate of the user request data.

[0188] In a possible implementation manner, based on the storage status of the target data corresponding to the target cache node and the storage status of the adjacent nodes of the target data corresponding to the target cache node, to determine the access status of the target data, the data prediction module is specifically configured to:

[0189] Based on the storage status of the target data corresponding to the target cache node and the storage status of the adjacent nodes of the target data corresponding to the target cache node, if it is determined that the target data is cached in any one of the cache nodes within a preset range, it is determined that the target data can be accessed by any one of the users; the cache nodes within the preset range are the target cache nodes corresponding to any one of the users and the adjacent nodes of the target cache node;

[0190] If it is determined that the target data is not cached in any one of the cache nodes within the preset range, it is determined that the target data is not accessed by any one of the users.

[0191] In a possible implementation manner, after determining the storage status of the target data in each cache node during the second period and before caching the target data into the corresponding cache node, there is also a data clearing module, which is used for:

[0192] Clear the data cached in each cache node, and cache the data of the same type as the target data in the newly added data into the cache node corresponding to the target data; the newly added data is the data received by the central server during the first period.

[0193] Next, refer to Figure 8 to describe the electronic device 130 according to this implementation manner of the present application. Figure 8 The displayed electronic device 130 is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0194] As Figure 8 shown, the electronic device 130 is presented in the form of a general-purpose electronic device. The components of the electronic device 130 may include but are not limited to: the at least one processor 131 described above, the at least one memory 132 described above, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0195] The bus 133 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, a processor bus, or a local bus using any of the multiple bus architectures.

[0196] The memory 132 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0197] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324. Such program modules 1324 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, and any one or some combination of these examples may include an implementation of a network environment.

[0198] The electronic device 130 may also communicate with one or more external devices 134 (such as a keyboard, a pointing device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or may communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (such as a router, a modem, etc.). Such communication may be through an input / output (I / O) interface 135. Further, the electronic device 130 may communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 136. As shown in the figure, the network adapter 136 communicates with other modules for the electronic device 130 through the bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0199] In an exemplary embodiment, the present application also provides a computer-readable storage medium including instructions, such as the memory 132 including instructions, and the above instructions can be executed by the processor 131 of the electronic device 130 to complete the above data caching method. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices, etc.

[0200] In an exemplary embodiment, there is also provided a computer program product including a computer program, and when the computer program is executed by the processor 131, it implements the data caching method provided by the present application.

[0201] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0202] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0203] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufacture including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0205] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A data caching method, characterized in that, The method includes: Based on the user request information of multiple data cached by multiple cache nodes within a first period, determining the frequency at which each user requests the target data cached by the multiple cache nodes within the first period; Based on the frequency at which each user requests the target data cached by the multiple cache nodes within the first period, determining the storage status of the target data at each cache node within a second period when the cache space occupied by the data cached at each cache node satisfies a first constraint condition and the average success rate of user-requested data satisfies a second constraint condition; wherein the second period is after the first period and adjacent to the first period; Based on the storage status of the target data at each cache node within the second period, caching the target data to the corresponding cache node.

2. The method according to claim 1, characterized in that, The first constraint condition is that the cache space occupied by the data cached at each cache node is not greater than the cache space capacity of each target cache node, and the second constraint condition is to maximize the average success rate of user-requested data.

3. The method according to claim 1, characterized in that, The determining the frequency at which each user requests the target data cached by the multiple cache nodes within the first period based on the user request information of multiple data cached by multiple cache nodes within the first period includes: For any one user, based on the user request information of multiple data cached by multiple cache nodes within the first period, determining the number of times the any one user requests the target data; and the sum of the number of times the any one user requests multiple target data; Based on the number of times the any one user requests the target data and the sum of the number of times the any one user requests multiple target data, determining the frequency at which the any one user requests the target data cached by the multiple cache nodes within the first period.

4. The method according to claim 1, wherein The determining the storage status of the target data at each cache node within the second period when the cache space occupied by the data cached at each cache node satisfies a first constraint condition and the average success rate of user-requested data satisfies a second constraint condition based on the frequency at which each user requests the target data cached by the multiple cache nodes within the first period includes: Based on the number of cache nodes, combining the storage statuses of the target data corresponding to each cache node to obtain multiple cache status sets, and summarizing the multiple cache status sets corresponding to the multiple target data to obtain a total cache status set, where each cache status set includes a storage status corresponding one-to-one to the cache nodes, and each storage status indicates whether the target data is cached to the corresponding cache node; Traversing each cache status set in the total cache status set and performing the following process: Based on the storage statuses in the traversed cache status sets and the frequency at which each user requests the target data cached by the multiple cache nodes within the first period, obtaining the average success rate of user-requested data; and based on the storage statuses in the traversed cache status sets, obtaining the cache space occupied by the multiple target data cached at each cache node. Determine from the cache state sets where the cache space occupied by the multiple pieces of target data cached in each cache node is not greater than the cache capacity of each cache node, and the cache state set with the highest average success rate of the corresponding user-requested data; Use the storage state in the selected cache state set as the storage state of the target data in each cache node during the second period.

5. The method according to claim 4, characterized in that, The obtaining of the average success rate of user-requested data based on the storage state in each of the traversed cache state sets and the frequencies of each user requesting the target data cached in the multiple cache nodes during the first period includes: For any user, determine the target cache node corresponding to the any user and the adjacent nodes of the target cache node, and determine the access state of the target data based on the storage state of the target data corresponding to the target cache node and the storage state of the target data corresponding to the adjacent nodes of the target cache node; the access state indicates whether the target data can be accessed by the any user; Based on the access state of the target data and the frequency of the any user requesting the target data during the first period, determine the request success rate of the any user for the target data; Perform a summation process on the multiple request success rates of the any user for multiple pieces of target data to obtain the request success rate of the any user for the requested data; Perform an averaging process on the sum of the request success rates of multiple pieces of the any user for the requested data corresponding to each user to obtain the average success rate of user-requested data.

6. The method according to claim 4, wherein The determining of the access state of the target data based on the storage state of the target data corresponding to the target cache node and the storage state of the target data corresponding to the adjacent nodes of the target cache node includes: Based on the storage state of the target data corresponding to the target cache node and the storage state of the target data corresponding to the adjacent nodes of the target cache node, if it is determined that the target data is cached in any one of the cache nodes within a preset range, determine that the target data can be accessed by the any user; the cache nodes within the preset range are the target cache node corresponding to the any user and the adjacent nodes of the target cache node; If it is determined that the target data is not cached in any one of the cache nodes within the preset range, determine that the target data cannot be accessed by the any user.

7. The method according to claim 1, characterized in that, After determining the storage state of the target data in each cache node during the second period and before caching the target data into the corresponding cache node, it further includes: Clear the data cached in each cache node, and cache the data of the same type as the target data in the newly added data into the cache node corresponding to the target data; the newly added data is the data received by the central server during the first period.

8. A data caching device, characterized in that, The device includes: A frequency determination module, configured to determine, based on user request information of multiple data cached by multiple cache nodes within a first period, the frequency at which each user requests the target data cached by the multiple cache nodes within the first period; A data prediction module, configured to determine, based on the frequency at which each user requests the target data cached by the multiple cache nodes within the first period, the storage status of the target data at each cache node within a second period when the cache space occupied by the data cached at each cache node meets a first constraint condition and the average success rate of user-requested data meets a second constraint condition; wherein the second period is after the first period and adjacent to the first period; A data sending module, configured to cache the target data into corresponding cache nodes based on the storage status of the target data at each cache node within the second period.

9. An electronic device, characterized in that, Comprising: A processor and a memory; The memory is configured to store executable instructions of the processor; The processor is configured to execute the instructions to implement the data caching method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the data caching method according to any one of claims 1-7.

Images