Cache policy determination method and apparatus, storage medium, program product, and computer device

Abstract

The application discloses a cache strategy determination method and device, a storage medium, a program product and a computer device. The method comprises the following steps: obtaining request access data, wherein the request access data comprises the number of times that a plurality of resources are requested to be accessed in M time periods, M is a positive integer; for each time period in the M time periods, for any two resources in the plurality of resources, based on the request access data, the number of common accesses of the any two resources in the time period is determined, and based on the number of common accesses and the number of times that any one resource in the any two resources is requested to be accessed in the time period, probability information of the any one resource in the time period is determined; based on the M time periods, the probability information and the request access data, a resource cache strategy is determined, so that the transmission of useless redundant resources by a CDN cluster can be reduced, the data processing capacity of the cluster is released, and the optimization effect of the CDN cluster on the network resource access speed is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, storage medium, program product, and computer device for determining a caching strategy. Background Technology

[0002] High-speed caching is a technical means in CDN (Content Delivery Network) services. CDN can deploy distributed nodes (usually called edge nodes or cache nodes) at the network edge and pre-cache resource data from the origin server to nodes closer to the user, thereby reducing the number of direct access requests from the origin server and improving the access speed of network resources.

[0003] In related technologies, when caching resources, it is usually necessary to periodically pull or push the full amount of resources within the configured range according to a preset time period. This results in a large number of useless and redundant resources being continuously transmitted in the CDN cluster over a large period of time, which occupies the data processing capacity of the entire cluster and weakens the optimization effect of the CDN cluster on network resource access speed. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes a caching strategy determination method, apparatus, storage medium, program product, and computer equipment, which can improve the optimization effect of CDN clusters in network resource access speed.

[0005] In a first aspect, embodiments of this application provide a method for determining a caching strategy, including: Obtain request access data, wherein the request access data includes the number of times each of multiple resources is requested to be accessed within M time periods, where M is a positive integer; For each of the M time periods, for any two resources among the plurality of resources, based on the requested access data, determine the common access count of any two resources being requested to be accessed within that time period, and based on the common access count and the number of times any one of the two resources is requested to be accessed within that time period, determine the probability information of any one resource within that time period. Based on the M time periods, the probability information, and the request access data, a resource caching strategy is determined.

[0006] Optionally, the probability information includes a second probability; The determination of the probability information of any one resource within the time period based on the number of common accesses and the number of times any one of the two resources is requested to be accessed within the time period includes: Based on the number of common accesses and the number of times any one of the two resources is requested to be accessed during the time period, the second probability of any one resource during the time period is determined by the Naive Bayes theorem. The second probability is used to characterize the probability that the other of the two resources will be accessed during the time period if any one resource has been accessed during the time period.

[0007] Optionally, determining the resource caching strategy based on the M time periods, the probability information, and the request access data includes: For each of the M time periods, based on the requested access data, determine the first probability that each of the multiple resources will be requested to be accessed within that time period. For each of the M time periods, a two-dimensional probability matrix is ​​determined based on the first probability and probability information of each of the multiple resources in that time period. The resource caching strategy is determined based on the M time periods and their corresponding two-dimensional probability matrices.

[0008] Optionally, the resource caching policy is used to indicate the resource caching queue; The process of determining the resource caching strategy based on the M time periods and their corresponding two-dimensional probability matrices includes: Based on the M time periods and their respective two-dimensional probability matrices, a three-dimensional probability matrix is ​​generated or updated. The three-dimensional probability matrix before or after the update contains n two-dimensional probability matrices, where n is a positive integer and n≤N, and N is the maximum number of two-dimensional matrices corresponding to the three-dimensional probability matrix. The resource cache queue is determined based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix.

[0009] Optionally, the n two-dimensional probability matrices correspond one-to-one with the n time periods; The updated three-dimensional probability matrix includes: When n < N and (n+M) ≤ N, the three-dimensional probability matrix before the update is updated based on the M time periods and their respective two-dimensional probability matrices. When n=N, or n<N and (n+M)>N, the effectiveness of each time period in the time period set is determined based on the distance of each time period from the current time in the time period set composed of the M time periods and the n time periods, and the weight associated with the two-dimensional probability matrix corresponding to each time period in the time period set. Furthermore, the three-dimensional probability matrix before the update is updated based on the two-dimensional probability matrices corresponding to the N time periods with the highest effectiveness in the time period set.

[0010] Optionally, determining the resource cache queue based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix includes: Based on each of the two-dimensional probability matrices, a target resource corresponding to each two-dimensional probability matrix and an associated resource corresponding to the target resource are determined. The multiple resources include the target resource and the associated resource. The first probability of the target resource in the time period corresponding to each two-dimensional probability matrix is ​​higher than a preset target resource probability threshold. When the associated resource and its corresponding target resource are any two resources, at least one of the two probability information corresponding to the two resources satisfies a preset probability condition. The two probability information refers to the probability information of the two resources in the time period corresponding to each two-dimensional probability matrix. The resource cache queue is determined based on the target resources and associated resources corresponding to each of the two-dimensional probability matrices.

[0011] Secondly, embodiments of this application provide a caching strategy determination apparatus, comprising: The data acquisition module is used to acquire request access data, wherein the request access data includes the number of times each of multiple resources is requested to be accessed within M time periods, where M is a positive integer; The probability information determination module is used to determine, for each of the M time periods, the common access count of any two resources among the plurality of resources based on the request access data, when both resources are requested to be accessed within the time period; and, based on the common access count and the number of times any one of the two resources is requested to be accessed within the time period, the probability information of any resource in the time period is determined. The strategy determination module is used to determine a resource caching strategy based on the M time periods, the probability information, and the request access data.

[0012] Thirdly, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the above-mentioned embodiments.

[0013] Fourthly, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the method described in any of the above-described embodiments.

[0014] Fifthly, embodiments of this application provide a computer device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the preceding claims.

[0015] In summary, the embodiments of this application have at least the following beneficial effects: By employing the embodiments of this application, request access data is obtained, wherein the request access data includes the number of times each of multiple resources is requested to be accessed within M time periods, where M is a positive integer; for each of the M time periods, for any two resources among the multiple resources, based on the request access data, the common access count of the two resources being requested to be accessed within that time period is determined, and based on the common access count and the number of times any one of the two resources is requested to be accessed within that time period, the probability information of that resource under that time period is determined; based on the M time periods, the probability information, and the request access data, a resource caching strategy is determined. In this way, considering the probability information of each resource under each time period, a corresponding resource caching strategy can be specifically determined, so that the strategy can conform to the statistical principles of the considered probability information, thereby reducing the transmission of useless redundant resources by the CDN cluster, releasing the data processing capacity of the cluster, and improving the optimization effect of the CDN cluster in network resource access speed. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the timestamp strategy provided by the relevant technology; Figure 2 This is a schematic diagram of the TTL strategy provided by related technologies; Figure 3 This is a schematic diagram of the path caching strategy provided by related technologies; Figure 4 This is a flowchart illustrating the caching strategy determination method provided in an embodiment of this application; Figure 5 This is a schematic diagram of the access probability tree construction provided in an embodiment of this application; Figure 6 This is a schematic diagram of a two-dimensional probability matrix provided in an embodiment of this application; Figure 7 This is a schematic diagram of the three-dimensional probability matrix queue provided in an embodiment of this application; Figure 8 This is a schematic diagram of the voting mechanism provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of the cache strategy determination device provided in the embodiments of this application; Figure 10This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments / examples are only a part of the embodiments / examples of this application, and not all of the embodiments / examples. Based on the embodiments / examples in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more. In the description of this application, the term "comprising" and its variations are open-ended, meaning "including but not limited to." The term "based on" means "at least partially based on." The term "according to" means "at least partially according to." The term "one embodiment / example" means "at least one embodiment / example"; the term "another embodiment / example" means "at least one additional embodiment / example"; the term "some embodiments / examples" means "at least some embodiments / examples."

[0019] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0020] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the application. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0021] In related technologies, CDN clusters can employ timestamp strategies, TTL (Time To Live) strategies, and path strategies to provide caching configurations for users when caching resources. Timestamp strategies cache resources based on the timestamp in the URL (Uniform Resource Locator). If the timestamp in the URL hasn't changed, the CDN server directly returns the locally cached file without needing to request the origin server again. TTL strategies cache resources based on their lifespan on the CDN server. If the resource hasn't expired, the CDN server can directly return the locally cached file. Path caching strategies involve customers pre-configuring paths to frequently used resources, allowing the CDN server to directly cache resources under these paths, thereby speeding up user access. This can be achieved by configuring specified directories, specifying file extensions, or using regular expressions to match a type of resource.

[0022] For timestamp strategies, please refer to Figure 1 The diagram illustrates a timestamp strategy provided by related technologies. When the timestamp strategy configured on a node takes effect, the edge node receives a user request, extracts the timestamp parameter attached to the request URL path, and compares it with the timestamp of the locally cached resource. If the two match or the timestamp has not exceeded the preset validity period, the request is determined to be still valid, and the local cached file is responded to directly; otherwise, the origin server process is triggered to obtain the updated resource from the origin server. This strategy relies on the timestamp in the URL as a version control basis, avoiding the problem of repeated origin server requests due to outdated resources. However, since the expiration time is an absolute time value, a large discrepancy between the server time and the client time will lead to caching errors, resulting in frequent origin server requests and affecting the acceleration effect.

[0023] The TTL (Time To Live) policy determines whether a resource can be cached based on its lifespan on the CDN server. See also... Figure 2 The diagram illustrates a TTL (Time-To-Live) strategy provided by related technologies. When a current request arrives at an edge node, it determines whether the local cached resource has expired. If it hasn't expired, the local cached file is returned directly; if it has expired, the origin server's cached resource needs to be periodically fetched to update the local copy. This strategy requires edge nodes to periodically perform fetch operations according to a preset cache expiration time. Therefore, the length of the cache expiration time directly affects the cache hit rate. If it's set too short, it will lead to frequent origin server requests, increasing network overhead; if it's set too long, users may not be able to obtain the latest resource data from the origin server for a long time. Furthermore, even if some resources are not accessed by users for a period of time, they will still continue to be transmitted, consuming the CDN cluster's acceleration capacity, resulting in the invalid transmission of a large amount of redundant resources and reducing overall resource utilization.

[0024] Path caching strategies refer to clients pre-configuring path rules for several frequently used resources, causing the CDN server to perform caching operations only on resources under the specified paths. See also... Figure 3 This diagram illustrates a path caching strategy provided by related technologies. When a user request arrives, the edge node determines whether the requested resource falls within a preset range of cached resource data. If a match is found, the local cached file is returned directly; otherwise, it needs to be retrieved from the origin server. This strategy is typically implemented by configuring specified directories, specifying file extensions, or using regular expression matching. However, because this method relies on a fixed preset list of cached files, the CDN acceleration effect will significantly decrease when users frequently request resources outside the preset range. Furthermore, the preset resources cannot dynamically adjust the cached content based on real-time user behavior, resulting in poor configuration flexibility and difficulty in adapting to changing access patterns.

[0025] It is evident that while the three caching strategies mentioned above each have their advantages, they all have significant limitations in actual deployment: the timestamp strategy is greatly affected by the time synchronization accuracy; the TTL strategy is prone to resource waste or update lag; and the path strategy lacks dynamic adaptability.

[0026] In view of this, embodiments of this application propose a method, apparatus, storage medium, program product, and computer device for determining a caching strategy, aiming to at least partially address the shortcomings of the aforementioned related technologies.

[0027] Firstly, see [the following] Figure 4 The diagram shows a flowchart of a caching strategy determination method provided in an embodiment of this application. This caching strategy determination method can be applied to a computer device with data processing capabilities. The method includes steps S401-S403, as detailed below.

[0028] S401, Obtain request access data, wherein the request access data includes the number of times each of the multiple resources is requested to be accessed within M time periods, where M is a positive integer.

[0029] In some examples, access request data may include the number of times each resource is requested to be accessed within each time period. For instance, access request data can originate from user access logs in a real network environment. This data can be used to record the number of times multiple resources are requested to be accessed within M time periods, where M is a positive integer. Specifically, in actual deployments, access request logs from the current user within the most recent preset period (e.g., the most recent hour) can be collected as access request data for statistical analysis of user behavior. The log file is divided into preset time periods (e.g., 5 minutes) as the smallest time granularity. The log files for each of the M time periods form a log package file. Resources accessed within the same log package file are considered "simultaneously accessed" or "jointly accessed" within that time period, meaning they belong to the same access session.

[0030] In some examples, request access data may include the following information for each time period: the URL path of the domain requested by the user, the number of requests corresponding to that URL path, and the access timestamp. By parsing this log data, the correlation between resources and access patterns can be further extracted.

[0031] In some examples, to improve path matching efficiency and tree structure construction accuracy during the parsing of request access data, efficient text processing techniques such as the KMP (Knuth-Morris-Pratt) string matching algorithm can be used to count the maximum common prefix (i.e., the maximum common path) of the request paths of various URL resources under the target domain. For example, for the URL path set { / static / js / app.js, / static / css / style.css, / static / img / logo.png}, its maximum repeating path is " / static". This maximum repeating path will serve as the root node for subsequently constructing the access probability tree, i.e., the root of the access probability tree. Based on this, a tree structure is built layer by layer downwards, where each node represents a path segment, and the edge weight can represent the request probability of that path segment under its parent node (i.e., the request probability of that path segment among paths at the same level). In this embodiment, the user's access behavior within a short period of time can be statistically analyzed based on the user's access log file. The maximum repeating path of the user's requested URL data can be parsed using algorithms such as KMP, and the access probability tree of the current user can be constructed. This can be obtained by statistically analyzing existing data without the need for real-time collection, thus reducing the data processing performance pressure on CDN cluster nodes.

[0032] S402, for each of the M time periods, for any two resources among the plurality of resources, based on the requested access data, determine the common access count of the two resources being requested to be accessed in the time period, and based on the common access count and the number of times any one of the two resources is requested to be accessed in the time period, determine the probability information of the resource in the time period.

[0033] In some examples, for each of the M time periods, for any two of the plurality of resources, based on the requested access data, the number of times these two resources are requested to be accessed within that time period is determined. The "number of times of joint access" can refer to the number of times two resources appear simultaneously in the access sessions of at least one user within the same time period (e.g., a 5-minute window). This number of joint accesses can be used to reflect the tendency for collaborative access between resources.

[0034] In some examples, probabilistic information can be determined by calculating conditional probabilities and / or joint probabilities. For instance, let C_AB be the number of times resource A and resource B are jointly accessed within a certain time period, and N_A be the total number of times resource A is accessed within that time period. Then, the conditional probability of resource B given that resource A is accessed can be defined as P(B|A) = C_AB / N_A. Similarly, P(A|B) and / or the joint probability P(A,B) = C_AB / T can be calculated, where T is the total number of sessions within that time period (e.g., the total number of times all resources are requested to be accessed within that time period). This type of probabilistic information can be used to characterize dependencies and access preferences between resources.

[0035] S403, Based on the M time periods, the probability information, and the request access data, determine the resource caching strategy.

[0036] In some examples, the strategy generation prompt, the M time periods, the probability information, and the request access data can be input into a large model to obtain a resource caching strategy output by the large model. The strategy generation prompt can be used to prompt the large model to use the probability information as an auxiliary prompt to generate a resource caching strategy based on the M time periods and the request access data.

[0037] In some examples, a pre-trained resource caching strategy determination model can be used to determine a resource caching strategy based on M time periods, probability information, and request access data. This model can be a trained model capable of predicting resource caching strategies using M time periods, probability information, and request access data as input and a resource caching strategy as output. During training, sample time periods, sample probability information, and sample request access data can be used as sample data (which also carries a corresponding expected strategy label representing the expected resource caching strategy). The predicted resource caching strategy generated by the model based on this sample data is obtained. Based on the difference between the predicted resource caching strategy and the expected resource caching strategy represented by the label, a general loss function is used to calculate the loss value. A general training algorithm (e.g., gradient descent) is then used to train the model based on this loss value, enabling the trained model to possess the aforementioned capabilities. For example, the model may include an input representation layer, a feature fusion and representation layer, and a prediction output layer. The input representation layer can receive data from the input model and convert the data into the desired feature vector form. For example, the input representation layer can use word embeddings or pre-trained language models (such as bidirectional language representation models based on the Transformer architecture) to generate semantic vectors, and / or use embedding layers to generate dense vectors. The feature fusion and representation layer can be used to fuse the feature vectors converted by the input representation layer. For example, the feature fusion and representation layer can implement the fusion through fully connected layers or attention mechanism layers. The prediction output layer can be used to generate prediction results based on the fused features. For example, the prediction output layer can use a softmax layer to output the probability distribution for different categories, and then output the prediction result based on the probability (for example, it can output the top one or more classification results with the highest probability as the prediction result).

[0038] In some examples, the caching strategy may include one or more of the following: which resources should be cached first, which resources should be cached in groups (based on high common access probability), how the cache hierarchy should be set (based on the path probability tree depth), and whether to use differentiated caching strategies for different time periods. For example, if a resource exhibits high access probability in multiple time periods and has a strong co-occurrence relationship with multiple other resources, then that resource can be included in the core cache set; if a path branch dominates in the probability tree (e.g., the request probability exceeds a threshold), then the entire subtree can be pre-cached.

[0039] In one optional implementation, the probability information includes a second probability; The determination of the probability information of any one resource within the time period based on the number of common accesses and the number of times any one of the two resources is requested to be accessed within the time period includes: Based on the number of common accesses and the number of times any one of the two resources is requested to be accessed during the time period, the second probability of any one resource during the time period is determined by the Naive Bayes theorem. The second probability is used to characterize the probability that the other of the two resources will be accessed during the time period if any one resource has been accessed during the time period.

[0040] In some examples, the second probability included in the probability information can be used to characterize the probability that, given that either of the two resources was accessed during the time period, the other resource will be accessed during the time period. Specifically, the second probability can be calculated using the Naive Bayes Theorem. According to the Naive Bayes Theorem, if resource A and resource B are associated, then given that resource B has been accessed, the posterior probability P(A|B) of resource A being accessed can be calculated using the following formula: P(A|B) = P(B|A) * P(A) / P(B) = N(A∩B) / N(B), where P(B|A) = N(A∩B) / N(A), N(A∩B) is the number of times resource A and resource B are accessed together in the same time period in the requested access data, N(A) is the number of times resource A is accessed in the requested access data, and N(B) is the number of times resource B is accessed in the requested access data. P(A|B) = N(A∩B) / N(B) indicates that, given that resource B has already been accessed, the probability of resource A being accessed is equal to the ratio of the number of times resource A and resource B are accessed together to the total number of times resource B is accessed. This probability value can be used to reflect the strength of dependencies between resources and can be used to guide the prefetching and priority sorting of the caching system.

[0041] In one optional implementation, determining the resource caching strategy based on the M time periods, the probability information, and the request access data includes: For each of the M time periods, based on the requested access data, determine the first probability that each of the multiple resources will be requested to be accessed within that time period. For each of the M time periods, a two-dimensional probability matrix is ​​determined based on the first probability and probability information of each of the multiple resources in that time period. The resource caching strategy is determined based on the M time periods and their corresponding two-dimensional probability matrices.

[0042] In some examples, the first probability of a single resource within any given time period can refer to the proportion of the number of times that resource is requested to be accessed within that time period out of the total number of resource requests within that time period, i.e., the unique access probability of that resource within that time period.

[0043] In some examples, see Figure 5 This diagram illustrates the construction of an access probability tree according to an embodiment of this application. The access probability tree is constructed based on the URL path data of the user's accessed resources in the request access data. Specifically, it can calculate the number of times the same path segment is repeatedly accessed in the request access data and the proportion of this number to the total number of accesses to the same level path segment. This proportion can be used as the prior probability P(A) of predicting the user's access request for that path segment, i.e., the first probability. Each node in the access probability tree can be a part or the entire URL path of the user's domain name. Leaf nodes can be a specific resource (a specific accessed resource data file) or a partial URL path (i.e., a directory, which may contain multiple resources). The type of leaf nodes in the access probability tree can be determined by a preset probability tree depth to avoid excessive growth of the probability tree during construction. For example, in... Figure 5 In this model, the root URL is the root node, and its branches are sub-URLs 1, 2, and 3, each corresponding to a different access path. The path segment connected to each child node and its access probability can be calculated based on the historical access frequency determined by the request access data. For example, the edge weight from sub-URL 1 to sub-URL 1 is 0.1, indicating that 10% of all requests accessing sub-URL 1 enter the sub-URL 1 path. In this embodiment, by constructing an access probability tree, caching decisions can be elevated from the single resource level to the path structure level. Fine-grained caching configuration can be performed based on the request probability of each node in the tree (e.g., JS files account for 80% of a directory). For example, full caching can be implemented for high-probability subtrees, while on-demand loading can be used for low-probability branches, optimizing storage and bandwidth utilization.

[0044] In some examples, see Figure 6 The diagram illustrates a two-dimensional probability matrix provided in an embodiment of this application. The two-dimensional probability matrix can be an n×n matrix, where n represents the number of resources involved in the time period. The matrix element P(i,j) represents the conditional probability that resource j is also accessed given that resource i is accessed within the time period, i.e., P(j|i). The construction process of this matrix is ​​as follows: The element P(i,i) on the main diagonal of the matrix represents the prior probability of resource i in that time period, i.e., the first probability. The off-diagonal element P(i,j) (i≠j) is calculated from the aforementioned conditional probability (i.e., the second probability).

[0045] Furthermore, the sample size corresponding to the requested access data can be used to determine the weights of the two-dimensional probability matrix, with the weights being positively correlated with the matrix's validity. The intersection points of rows and columns represent probability values, the diagonal lines of the matrix represent prior probabilities (first probabilities), and the other points represent posterior probabilities (second probabilities). For example, in... Figure 6 In the matrix, the element P(B|A) represents the probability that resource B will be accessed given that resource A has been accessed. This value is derived from the ratio of the number of times resource A and resource B are accessed simultaneously to the number of times resource A is accessed in the statistical sample data.

[0046] Furthermore, if more complex conditional probabilities need to be calculated, such as the probability P(C|A∩B) that a user accesses resource A and resource B within a short period of time and then accesses resource C, it can be extended to a three-dimensional structure. In this case, the probability matrix will have a three-dimensional structure, and its calculation formula is: P(C|AB)=N(A∩B∩C) / N(A∩B), where N(A∩B∩C) is the number of times resource A, resource B, and resource C are all accessed within this time period.

[0047] In one alternative implementation, the resource caching policy is used to indicate a resource caching queue; The process of determining the resource caching strategy based on the M time periods and their corresponding two-dimensional probability matrices includes: Based on the M time periods and their respective two-dimensional probability matrices, a three-dimensional probability matrix is ​​generated or updated. The three-dimensional probability matrix before or after the update contains n two-dimensional probability matrices, where n is a positive integer and n≤N, and N is the maximum number of two-dimensional matrices corresponding to the three-dimensional probability matrix. The resource cache queue is determined based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix.

[0048] In some examples, if a three-dimensional probability matrix has not been generated beforehand, the generated three-dimensional probability matrix can be formed by concatenating the two-dimensional probability matrices corresponding to the M time periods in chronological order.

[0049] In some examples, a three-dimensional probability matrix can be composed of multiple two-dimensional probability matrices. The structure of the three-dimensional probability matrix uses time as the third dimension, that is, two-dimensional probability matrices from different time periods are concatenated in chronological order to form a probability matrix queue, which serves as the three-dimensional probability matrix. See also... Figure 7The diagram illustrates a three-dimensional probability matrix queue provided in an embodiment of this application. The number of two-dimensional matrices stored in the queue gradually increases over time. When the queue length reaches a preset maximum value N, if the validity of a newly constructed two-dimensional matrix is ​​greater than the validity of any matrix in the current queue, the new matrix is ​​enqueued, and older matrices with lower validity than the new matrix are dequeued and become invalid. The validity of each two-dimensional probability matrix is ​​related to its validity period, which decays after a specified period (e.g., exponentially). Furthermore, the decay exponent of the validity of matrices in the queue gradually decreases over time to ensure that recent data has a higher weight.

[0050] In some examples, a resource cache queue can be used to indicate which of the aforementioned resources should be cached. The resource cache queue can contain at least one resource record corresponding one-to-one with each resource to be cached. Each resource record has a voting attribute, and the number of votes for each resource record can be determined by the various two-dimensional probability matrices in the three-dimensional probability matrix queue. Specifically, a two-dimensional probability matrix can be associated with at least one resource that the current user is most likely to access, and one vote is allocated to each of these at least one resources. Voting is isolated between multiple two-dimensional probability matrices; that is, the same resource can be associated with multiple two-dimensional probability matrices and obtain multiple votes corresponding one-to-one with each of the multiple two-dimensional probability matrices. This embodiment can achieve cumulative evaluation of resource access probability to improve the accuracy and robustness of caching decisions.

[0051] In some examples, resource cache queues can be used to indicate resource caching strategies, where resources in the queue are ordered based on the total number of votes a resource receives in the current three-dimensional probability matrix queue. The higher the total number of votes, the higher the priority, and the more likely it should be cached.

[0052] In one optional implementation, the n two-dimensional probability matrices correspond one-to-one with the n time periods; The updated three-dimensional probability matrix includes: When n < N and (n+M) ≤ N, the three-dimensional probability matrix before the update is updated based on the M time periods and their respective two-dimensional probability matrices. When n=N, or n<N and (n+M)>N, the effectiveness of each time period in the time period set is determined based on the distance of each time period from the current time in the time period set composed of the M time periods and the n time periods, and the weight associated with the two-dimensional probability matrix corresponding to each time period in the time period set. Furthermore, the three-dimensional probability matrix before the update is updated based on the two-dimensional probability matrices corresponding to the N time periods with the highest effectiveness in the time period set.

[0053] In some examples, the effectiveness of each time period in the time period set can be determined by the product of the weights associated with the two-dimensional probability matrix corresponding to that time period and the time decay index. The time decay index is determined by the distance of that time period from the current time, and the weights can be determined by the total number of times all resources within that time period are accessed, as indicated by the associated two-dimensional probability matrix.

[0054] For example, the time decay index It can be determined using the following formula:

[0055] In the formula, The decay value corresponds to the effective value of the two-dimensional probability matrix. The initial value can be the same as... equal, The queue length of the three-dimensional probability matrix (the number of two-dimensional probability matrices contained in the three-dimensional probability matrix). This is the attenuation factor corresponding to the effective value of the two-dimensional matrix, and its initial value can be 0.

[0056] For example, the recursive update rules for the attenuation value and attenuation factor can be expressed as follows:

[0057]

[0058] Here, whenever a new two-dimensional probability matrix is ​​added to the three-dimensional probability matrix (i.e., in each round of updates), all existing matrices... The value will attempt to decrease by 1, but will not decrease to 0. Once it decreases to 1 (i.e., the next step will be to become 0), it will no longer decrease and will be fixed at 1. This allows the recursive update rules of the decay value and decay factor to be affected by the distance of the corresponding time period from the current time.

[0059] That is, over time, the decay exponent of the validity of matrices in the queue gradually decreases, and the validity period of a matrix can decay exponentially after exceeding a preset value. Each time a new two-dimensional probability matrix is ​​enqueued, the time decay exponent of existing matrices in the queue will become smaller and smaller. The product of the weight of the two-dimensional matrix and the time decay exponent is taken as the effective value of the matrix. and according to Determining whether a new matrix is ​​active and an old matrix is ​​inactive in the queue can further increase or decrease the number of each resource ticket associated in the cache queue indicated by the three-dimensional probability matrix.

[0060] In one optional implementation, determining the resource cache queue based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix includes: Based on each of the two-dimensional probability matrices, a target resource corresponding to each two-dimensional probability matrix and an associated resource corresponding to the target resource are determined. The multiple resources include the target resource and the associated resource. The first probability of the target resource in the time period corresponding to each two-dimensional probability matrix is ​​higher than a preset target resource probability threshold. When the associated resource and its corresponding target resource are any two resources, at least one of the two probability information corresponding to the two resources satisfies a preset probability condition. The two probability information refers to the probability information of the two resources in the time period corresponding to each two-dimensional probability matrix. The resource cache queue is determined based on the target resources and associated resources corresponding to each of the two-dimensional probability matrices.

[0061] In some examples, the first probabilities corresponding to each two-dimensional probability matrix can be determined from each two-dimensional probability matrix contained in the updated or generated three-dimensional probability matrix. Then, the first probability whose probability value is higher than the preset target resource probability threshold can be selected from the first probabilities, and the resource corresponding to the selected first probability can be used as the target resource corresponding to the two-dimensional probability matrix.

[0062] Furthermore, since when the target resource is taken as any one of the aforementioned two resources, any other resource among the multiple resources can be taken as the other one of those two resources, multiple probability information corresponding to the target resource can be obtained (which may include the probability information of the target resource in a corresponding time period, and may also include the probability information of the other resource in a corresponding time period). Then, the resources corresponding to at least one probability information among the multiple probability information corresponding to the target resource that satisfies a preset probability condition are respectively taken as the associated resources corresponding to the target resource. For example, when the probability information includes a second probability, the preset probability condition may include the second probability being higher than a preset associated resource probability threshold.

[0063] In some examples, see Figure 8This diagram illustrates the voting mechanism provided in this application embodiment. The resource caching queue serves as the basis for the current CDN node to cache origin server resources. The probability values ​​(i.e., the first probability) on the diagonal of the two-dimensional probability matrix that are greater than a preset target resource probability threshold represent the target resources that the current user is most likely to access. Furthermore, based on the selected target resources, associated resources with a posterior probability (i.e., the second probability) greater than a preset index value can be extracted from the corresponding rows and columns. Target resources and associated resources can form the resources required for caching in the resource caching queue. That is, the CDN node can periodically cache the data of the required cached resources based on the resource records in the resource caching queue. In addition, as time progresses, the addition and expiration of the two-dimensional probability matrix in the three-dimensional probability matrix queue can be updated synchronously. When the voting value of a resource record is less than or equal to 0, the record is removed from the queue. The CDN node does not need to continue caching resource data related to the removed record until a new two-dimensional matrix determines that the resource is the resource data that the current user is most likely to access. Only then is the resource record re-added to the resource caching queue, enabling the CDN node to flexibly and dynamically adjust the cached target resource files according to the current user's access situation.

[0064] Thus, the embodiments of this application may have at least one of the following beneficial effects.

[0065] (1) Significantly improve CDN cache hit rate: By introducing the Naive Bayes theorem to calculate the second probability, the conditional dependencies between resources can be accurately characterized. Not only are frequently accessed target resources cached, but also their highly related resources are prefetched simultaneously, so that resources are ready before the actual user request occurs, thus greatly improving the cache hit rate.

[0066] (2) Effectively reduce redundant resource transmission: Caching strategies in related technologies often cache a large number of low-frequency or isolated resources due to static rules or global average assumptions, resulting in bandwidth and storage waste. The embodiments of this application can cache resources that meet the first probability threshold and the association probability condition, avoiding the invalid distribution of useless data in the CDN cluster and reducing network load.

[0067] (3) Achieving dynamic and flexible cache adjustment: Through the three-dimensional probability matrix queue mechanism, the embodiments of this application can continuously update the cached content according to the user's latest access behavior. Recent behavior is given higher weight, and old data is automatically weakened as time decays, improving the fit between the caching strategy and the current user needs.

[0068] (4) Support for long-term learning and continuous optimization: With the accumulation of historical data and algorithm iteration, the embodiments of this application can continuously deepen the understanding of user access patterns. Through long-term machine learning and algorithm training, the accuracy of identifying target resources and related resources can be further improved, the cache hit rate can be continuously increased, and invalid cache items can be gradually eliminated.

[0069] (5) Improve the overall acceleration efficiency of CDN cluster: Since the cached content is closer to the real user needs, the number of requests to the origin is significantly reduced, the response speed of edge nodes is faster, the user experience is improved, and the pressure on the origin site is reduced. The resource utilization efficiency and acceleration performance of the entire CDN system are comprehensively improved.

[0070] Secondly, correspondingly, this application also provides a cache strategy determination device, which can implement all the processes of the cache strategy determination method provided in the above embodiments.

[0071] See Figure 9 The diagram shows a schematic of the cache policy determination device 900 provided in an embodiment of this application. The cache policy determination device 900 includes: The data acquisition module 901 is used to acquire request access data, wherein the request access data includes the number of times each of multiple resources is requested to be accessed within M time periods, where M is a positive integer; The probability information determination module 902 is used to determine, for each of the M time periods, the common access count of any two resources among the multiple resources based on the request access data, when both resources are requested to be accessed in the time period, and to determine the probability information of any resource in the time period based on the common access count and the number of times any one of the two resources is requested to be accessed in the time period. The strategy determination module 903 is used to determine a resource caching strategy based on the M time periods, the probability information, and the request access data.

[0072] In one optional implementation, the probability information includes a second probability; The determination of the probability information of any one resource within the time period based on the number of common accesses and the number of times any one of the two resources is requested to be accessed within the time period includes: Based on the number of common accesses and the number of times any one of the two resources is requested to be accessed during the time period, the second probability of any one resource during the time period is determined by the Naive Bayes theorem. The second probability is used to characterize the probability that the other of the two resources will be accessed during the time period if any one resource has been accessed during the time period.

[0073] In one optional implementation, determining the resource caching strategy based on the M time periods, the probability information, and the request access data includes: For each of the M time periods, based on the requested access data, determine the first probability that each of the multiple resources will be requested to be accessed within that time period. For each of the M time periods, a two-dimensional probability matrix is ​​determined based on the first probability and probability information of each of the multiple resources in that time period. The resource caching strategy is determined based on the M time periods and their corresponding two-dimensional probability matrices.

[0074] In one alternative implementation, the resource caching policy is used to indicate a resource caching queue; The process of determining the resource caching strategy based on the M time periods and their corresponding two-dimensional probability matrices includes: Based on the M time periods and their respective two-dimensional probability matrices, a three-dimensional probability matrix is ​​generated or updated. The three-dimensional probability matrix before or after the update contains n two-dimensional probability matrices, where n is a positive integer and n≤N, and N is the maximum number of two-dimensional matrices corresponding to the three-dimensional probability matrix. The resource cache queue is determined based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix.

[0075] In one optional implementation, the n two-dimensional probability matrices correspond one-to-one with the n time periods; The updated three-dimensional probability matrix includes: When n < N and (n+M) ≤ N, the three-dimensional probability matrix before the update is updated based on the M time periods and their respective two-dimensional probability matrices. When n=N, or n<N and (n+M)>N, the effectiveness of each time period in the time period set is determined based on the distance of each time period from the current time in the time period set composed of the M time periods and the n time periods, and the weight associated with the two-dimensional probability matrix corresponding to each time period in the time period set. Furthermore, the three-dimensional probability matrix before the update is updated based on the two-dimensional probability matrices corresponding to the N time periods with the highest effectiveness in the time period set.

[0076] In one optional implementation, determining the resource cache queue based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix includes: Based on each of the two-dimensional probability matrices, a target resource corresponding to each two-dimensional probability matrix and an associated resource corresponding to the target resource are determined. The multiple resources include the target resource and the associated resource. The first probability of the target resource in the time period corresponding to each two-dimensional probability matrix is ​​higher than a preset target resource probability threshold. When the associated resource and its corresponding target resource are any two resources, at least one of the two probability information corresponding to the two resources satisfies a preset probability condition. The two probability information refers to the probability information of the two resources in the time period corresponding to each two-dimensional probability matrix. The resource cache queue is determined based on the target resources and associated resources corresponding to each of the two-dimensional probability matrices.

[0077] Thirdly, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the above-mentioned embodiments.

[0078] Fourthly, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the method described in any of the above-described embodiments.

[0079] Fifthly, embodiments of this application provide a computer device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the preceding claims.

[0080] See Figure 10 The computer device in this embodiment includes a processor 1001, a memory 1002, and a computer program stored in the memory 1002 and executable on the processor 1001, such as a cache policy determination program. When the processor 1001 executes the computer program, it implements the steps in the various cache policy determination method embodiments described above, for example... Figure 4 The steps S401-S403 are shown.

[0081] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 1002 and executed by the processor 1001 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device.

[0082] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device may include, but is not limited to, a processor 1001 and a memory 1002. Those skilled in the art will understand that the schematic diagram is merely an example of a computer device and does not constitute a limitation on the computer device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0083] The processor 1001 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 1001 can be any conventional processor. The processor 1001 is the control center of the computer device, connecting various parts of the entire computer device through various interfaces and lines.

[0084] The memory 1002 can be used to store the computer programs and / or modules. The processor 1001 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 1002 and calling the data stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.), etc. In addition, the memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0085] Wherein, if the modules / units integrated into the computer device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a non-transitory computer-readable storage medium. When the computer program is executed by the processor 1001, it can implement the steps of the various method embodiments described above. Wherein, the computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0086] In summary, the embodiments of this application have at least the following beneficial effects: By employing the embodiments of this application, request access data is obtained, wherein the request access data includes the number of times each of multiple resources is requested to be accessed within M time periods, where M is a positive integer; for each of the M time periods, for any two resources among the multiple resources, based on the request access data, the common access count of the two resources being requested to be accessed within that time period is determined, and based on the common access count and the number of times any one of the two resources is requested to be accessed within that time period, the probability information of that resource under that time period is determined; based on the M time periods, the probability information, and the request access data, a resource caching strategy is determined. In this way, considering the probability information of each resource under each time period, a corresponding resource caching strategy can be specifically determined, so that the strategy can conform to the statistical principles of the considered probability information, thereby reducing the transmission of useless redundant resources by the CDN cluster, releasing the data processing capacity of the cluster, and improving the optimization effect of the CDN cluster in network resource access speed.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware platforms, or it can be implemented entirely by hardware. Based on this understanding, all or part of the technical solutions of this application that contribute to the background technology can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0088] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A method for determining a caching strategy, characterized in that, include: Obtain request access data, wherein the request access data includes the number of times each of multiple resources is requested to be accessed within M time periods, where M is a positive integer; For each of the M time periods, for any two resources among the plurality of resources, based on the requested access data, determine the common access count of any two resources being requested to be accessed within that time period, and based on the common access count and the number of times any one of the two resources is requested to be accessed within that time period, determine the probability information of any one resource within that time period. Based on the M time periods, the probability information, and the request access data, a resource caching strategy is determined.

2. The method according to claim 1, characterized in that, The probability information includes a second probability; The determination of the probability information of any one of the two resources within the time period, based on the number of common accesses and the number of times any one of the two resources is requested to be accessed within the time period, includes: Based on the number of common accesses and the number of times any one of the two resources is requested to be accessed during the time period, the second probability of any one resource during the time period is determined by the Naive Bayes theorem. The second probability is used to characterize the probability that the other of the two resources will be accessed during the time period if any one resource has been accessed during the time period.

3. The method according to any one of claims 1-2, characterized in that, The step of determining a resource caching strategy based on the M time periods, the probability information, and the request access data includes: For each of the M time periods, based on the requested access data, determine the first probability that each of the multiple resources will be requested to be accessed within that time period. For each of the M time periods, a two-dimensional probability matrix is ​​determined based on the first probability and probability information of each of the multiple resources in that time period. The resource caching strategy is determined based on the M time periods and their corresponding two-dimensional probability matrices.

4. The method according to claim 3, characterized in that, The resource caching policy is used to indicate the resource caching queue; The process of determining the resource caching strategy based on the M time periods and their corresponding two-dimensional probability matrices includes: Based on the M time periods and their respective two-dimensional probability matrices, a three-dimensional probability matrix is ​​generated or updated. The three-dimensional probability matrix before or after the update contains n two-dimensional probability matrices, where n is a positive integer and n≤N, and N is the maximum number of two-dimensional matrices corresponding to the three-dimensional probability matrix. The resource cache queue is determined based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix.

5. The method according to claim 4, characterized in that, The n two-dimensional probability matrices correspond one-to-one with the n time periods; The updated three-dimensional probability matrix includes: When n < N and (n+M) ≤ N, the three-dimensional probability matrix before the update is updated based on the M time periods and their respective two-dimensional probability matrices. When n=N, or n<N and (n+M)>N, the effectiveness of each time period in the time period set is determined based on the distance of each time period from the current time in the time period set composed of the M time periods and the n time periods, and the weight associated with the two-dimensional probability matrix corresponding to each time period in the time period set. Furthermore, the three-dimensional probability matrix before the update is updated based on the two-dimensional probability matrices corresponding to the N time periods with the highest effectiveness in the time period set.

6. The method according to claim 4, characterized in that, The step of determining the resource cache queue based on the various two-dimensional probability matrices contained in the updated or generated three-dimensional probability matrix includes: Based on each of the two-dimensional probability matrices, a target resource corresponding to each two-dimensional probability matrix and an associated resource corresponding to the target resource are determined. The multiple resources include the target resource and the associated resource. The first probability of the target resource in the time period corresponding to each two-dimensional probability matrix is ​​higher than a preset target resource probability threshold. When the associated resource and its corresponding target resource are any two resources, at least one of the two probability information corresponding to the two resources satisfies a preset probability condition. The two probability information refers to the probability information of the two resources in the time period corresponding to each two-dimensional probability matrix. The resource cache queue is determined based on the target resources and associated resources corresponding to each of the two-dimensional probability matrices.

7. A caching strategy determination device, characterized in that, include: The data acquisition module is used to acquire request access data, wherein the request access data includes the number of times each of multiple resources is requested to be accessed within M time periods, where M is a positive integer; The probability information determination module is used to determine, for each of the M time periods, the common access count of any two resources among the plurality of resources based on the request access data, when both resources are requested to be accessed within the time period; and, based on the common access count and the number of times any one of the two resources is requested to be accessed within the time period, the probability information of any resource in the time period is determined. The strategy determination module is used to determine a resource caching strategy based on the M time periods, the probability information, and the request access data.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method described in any one of claims 1-6.

10. A computer device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method of any one of claims 1-6.

Images