Caching method and device, electronic equipment, readable storage medium and program product

Abstract

The invention provides a caching method and device, electronic equipment, a readable storage medium and a program product, and can be applied to the technical field of big data and the technical field of artificial intelligence. The caching method comprises the steps that on the basis of an access log of cached data, feature elements of the cached data are determined, and the feature elements comprise service labels and access distribution feature data capable of representing the accessed condition of the cached data; constructing vector data points of the cache data based on the cache data, the service labels and the access distribution feature data; performing clustering analysis on the vector data points to obtain a cluster; and according to a comparison result of the cluster feature data of the clustering cluster and a preset threshold value, adjusting a caching strategy of the cached data, and caching the cached data according to the adjusted caching strategy.

Description

Technical Field

[0001] This application relates to the fields of big data technology and artificial intelligence technology, and specifically to a caching method, apparatus, electronic device, readable storage medium and program product. Background Technology

[0002] In the field of computer technology, caching refers to a temporary storage area in a computer system for frequently accessed data or computation results. Caching technology is used to improve data access speed and reduce the access burden on main storage or databases. Current caching strategies mainly rely on fixed eviction rules. Such caching strategies lack the ability to respond to dynamic business loads or changing data access patterns. Cached data cannot be updated accordingly to dynamic business loads or changing data access patterns, thus reducing the utilization rate of cache resources. Summary of the Invention

[0003] In view of the above problems, this application provides caching methods, apparatus, electronic devices, readable storage media and program products for improving the utilization of cache resources.

[0004] One aspect of this application provides a caching method, comprising: determining characteristic elements of the cached data based on access logs of the cached data, wherein the characteristic elements include business tags and access distribution characteristic data that can represent the access status of the cached data; constructing vector data points of the cached data based on the cached data, business tags, and access distribution characteristic data; performing cluster analysis on the vector data points to obtain clusters; adjusting the caching strategy of the cached data according to the comparison result of the cluster characteristic data of the clusters with a preset threshold, and caching the cached data according to the adjusted caching strategy.

[0005] According to an embodiment of this application, cluster analysis is performed on vector data points to obtain clusters, including: traversing the vector data points and forming a first neighborhood with each vector data point as the center and a predetermined neighborhood radius, wherein the first neighborhood includes a central vector data point as the center and neighboring vector data points that have a neighbor relationship with the central vector data point; constructing clusters based on the relationship between the number of vector data points in the first neighborhood and the predetermined number of vector data points, wherein the number of vector data points in the first neighborhood is obtained based on the number of neighboring vector data points.

[0006] According to an embodiment of this application, a cluster is constructed based on the relationship between the number of vector data points in the first neighborhood and a predetermined number of vector data points, including: in response to the fact that the number of vector data points in the first neighborhood is not less than the predetermined number of vector data points, marking the center vector data point as the first cluster center; constructing an initial cluster based on the first cluster center and the neighbor vector data points in the first neighborhood; for each neighbor vector data point in the initial cluster, if the neighbor vector data point is the second cluster center, adding a second neighborhood based on the second cluster center to the initial cluster to obtain a cluster, wherein the vector data points in the second neighborhood are density-reachable points of the first cluster center, and the second cluster center is determined when the number of vector data points in the second neighborhood is not less than the predetermined number of vector data points.

[0007] According to embodiments of this application, the preset thresholds include a first access rate value, a second access rate threshold, and a third access rate threshold, wherein the first access rate threshold is greater than the second access rate threshold, and the second access rate threshold is greater than the third access rate threshold; the caching strategy for cached data is adjusted based on the comparison result between the cluster feature data of the cluster and the preset thresholds, including: in response to the cluster feature data being greater than or equal to the first access rate threshold, extending the expiration time of cached data within the cluster and preloading the cached data; in response to the cluster feature data being greater than or equal to the second access rate threshold and less than the first access rate threshold, periodically refreshing the cached data within the cluster; in response to the cluster feature data being greater than or equal to the third access rate threshold and less than the second access rate threshold, shortening the expiration time of cached data within the cluster; and in response to the cluster feature data being less than the third access rate threshold, dynamically caching the cached data based on the relationship between the number of accesses to the cached data within the cluster in a predetermined time window and a predetermined number of accesses.

[0008] According to an embodiment of this application, the access log is an access log within a predetermined time period. Based on the access log of cached data, the characteristic elements of the cached data are determined, including: determining the total number of accesses to the cached data within the predetermined time period based on the accessed identifier in the access log, and obtaining the access frequency of the cached data based on the ratio between the total number of accesses and the predetermined time period; determining the standard deviation of the access time distribution and the concentration of the access time period of the cached data based on the access timestamp in the access log; determining the login dependency of the cached data based on the field in the access log used to indicate access after login; determining the number of objects accessing the cached data based on the object identifier in the access log, and obtaining the object proportion of the cached data based on the ratio between the number of objects and the total number of objects within the predetermined time period; and obtaining access distribution characteristic data based on the access frequency, standard deviation of the access time distribution, concentration of the access time period, login dependency, and object proportion of the cached data.

[0009] According to an embodiment of this application, determining the characteristic elements of cached data based on access logs of cached data includes: extracting source fields from the access logs, wherein the source fields can represent the request source of the cached data, and the source fields include at least one of the following: cache hit path of cached data, application programming interface request source, and page identifier; selecting target fields from the source fields according to the preset priority of the source fields; and generating business tags according to the business tag generation strategy corresponding to the target fields.

[0010] According to an embodiment of this application, the method further includes: updating vector data points to obtain updated vector data points when the changing trend of access distribution feature data is detected to meet a predetermined clustering reconstruction condition; performing clustering analysis based on the updated vector data points to obtain multiple updated clusters; and caching cached data based on the comparison result of the updated cluster feature data of the updated clusters with a preset threshold.

[0011] One aspect of this application also provides a caching device, comprising: a determining module, configured to determine characteristic elements of the cached data based on access logs of the cached data, wherein the characteristic elements include business tags and access distribution characteristic data that can represent the access status of the cached data; a constructing module, configured to construct vector data points of the cached data based on the cached data, business tags, and access distribution characteristic data; a first clustering module, configured to perform cluster analysis on the vector data points to obtain clusters; and a first caching module, configured to adjust the caching strategy of the cached data according to the comparison result of the cluster characteristic data of the clusters with a preset threshold, and cache the cached data according to the adjusted caching strategy.

[0012] Another aspect of this application provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above-described method.

[0013] Another aspect of this application provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0014] Another aspect of this application provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0015] According to the caching method, apparatus, electronic device, readable storage medium, and program product provided in the embodiments of this application, the characteristic elements of cached data are determined based on access logs of cached data; vector data points are constructed based on cached data, business tags, and access distribution characteristic data; cluster analysis is performed on the vector data points to obtain clusters; the caching strategy is adjusted based on the comparison results of the cluster characteristic data of the clusters with preset thresholds, and the cached data is cached according to the adjusted caching strategy. Because the caching process constructs vector data points of cached data based on multi-dimensional data such as cached data, business tags, and access distribution characteristic data, performs cluster analysis on the vector data points, and dynamically adjusts the caching strategy based on the analysis results, the cached data can flexibly change according to the dynamic actual access situation, optimizing the utilization rate of cache resources and achieving the technical effect of improving cache resource utilization. Attached Figure Description

[0016] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0017] Figure 1 This diagram illustrates an application scenario of the caching method according to an embodiment of this application.

[0018] Figure 2 A flowchart illustrating a caching method according to an embodiment of this application is shown schematically;

[0019] Figure 3 An architectural diagram of a caching system according to an embodiment of this application is illustrated schematically;

[0020] Figure 4 A schematic diagram illustrating the structure of a caching device according to an embodiment of this application is shown; and

[0021] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a caching method according to an embodiment of this application. Detailed Implementation

[0022] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0024] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0025] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0026] It should be noted that the caching method and apparatus provided in this application can be used in the fields of big data technology and artificial intelligence technology, as well as in any field other than big data technology and artificial intelligence technology. This application does not limit the application field of the caching method and apparatus.

[0027] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to cached data, access logs, data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0028] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this application all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0029] Currently, commonly used caching frameworks employ several common eviction strategies. One strategy is based on data access, prioritizing the eviction of data that hasn't been accessed for the longest time. Another is based on time expiration, where a time-limited expiration period or time is set for cached data, automatically deleting it upon reaching this expiration time. A third strategy is random eviction, which randomly deletes cached items when the caching system's memory reaches its limit. While these strategies have played a significant role in improving caching efficiency and reducing database load, they also share some common drawbacks, especially when dealing with complex or dynamic data access patterns.

[0030] Data access-based eviction policies typically free up space by removing the least accessed data, but their implementation is not always perfect. While data access-based eviction policies can effectively reduce memory pressure in many cases, they cannot be precisely optimized for all situations. For example, in high-concurrency or large-scale data environments, data access-based eviction policies may result in inaccurate eviction due to high computational costs or approximate implementations, potentially leading to the wrong eviction of important data, thus reducing cache hit rates and increasing the burden on database queries.

[0031] Time-based expiration cache management strategies control the cache lifecycle by setting a fixed expiration time for each cached item. While this strategy is simple and easy to understand, and ensures that expired data is cleared promptly, a fixed expiration time often cannot be dynamically adjusted based on the actual access frequency and change patterns of the data. For some cached data, the expiration time may be set too short, leading to frequent cache invalidation and repeated loading, wasting system resources; while for other cached data, the cache may be over-retained, occupying too much memory space and wasting valuable cache capacity. In addition, a fixed expiration time may cause frequently changing data to expire too early, resulting in a decrease in cache hit rate.

[0032] Random eviction is another simplified approach. When the cache system reaches its memory limit, it randomly evicts a portion of the data. While this method is simple to implement and consumes fewer resources, its drawback is that it completely disregards the access frequency or importance of the data. This could lead to the indiscriminate deletion of critical data, reducing cache efficiency and even negatively impacting application performance.

[0033] The aforementioned existing strategies generally share a common drawback: they are static and lack the ability to dynamically respond to data access patterns and business needs. Specifically, these strategies do not consider the frequency and trends of data access; cache expiration times and eviction mechanisms are based on simple, pre-set rules rather than adjustments made according to actual access patterns. This can lead to unsatisfactory resource utilization and hit rates in the caching system, resulting in frequent database accesses and unnecessary performance losses.

[0034] In view of this, embodiments of this application provide a caching method to address the problems existing in the above-mentioned strategies, such as the use of fixed expiration times, which are difficult to dynamically adjust according to changes in data access frequency, leading to the premature expiration of frequently accessed data and excessive memory consumption of infrequently accessed data. It also addresses the issue of poor performance under high concurrency and complex access modes, failing to consider the actual value and changes of data, potentially mistakenly evicting important data, thereby reducing cache hit rate. Furthermore, fixed caching strategies are difficult to effectively optimize memory resource usage, resulting in frequent cache refreshes and database queries, wasting the resource utilization of the caching system. This application introduces a clustering analysis algorithm, combined with data access patterns and trends, to dynamically adjust the caching strategy for cached data (e.g., adjusting expiration time and update mechanism), thereby improving cache hit rate and memory utilization efficiency, reducing unnecessary database access, and ensuring data consistency. The caching method provided by embodiments of this application is not only applicable to application scenarios with high concurrency and high real-time requirements, such as mobile banking, but also provides a more efficient and flexible cache management solution under complex business needs.

[0035] Specifically, the caching method provided in this application includes: determining the feature elements of the cached data based on the access logs of the cached data, wherein the feature elements include business tags and access distribution feature data that can represent the access status of the cached data; constructing vector data points of the cached data based on the cached data, business tags and access distribution feature data; performing cluster analysis on the vector data points to obtain clusters; adjusting the caching strategy of the cached data according to the comparison result of the cluster feature data of the clusters with a preset threshold, and caching the cached data according to the adjusted caching strategy.

[0036] Figure 1 The diagram illustrates an application scenario of the caching method according to an embodiment of this application.

[0037] like Figure 1As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, a server 105, and a caching system 106. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105, and between the server 105 and the caching system 106. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0038] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, such as sending access requests to cached data in the caching system 106, or receiving information about caching policy adjustment results. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as applications for managing the caching system, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (these are just examples).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0040] Server 105 can be a server providing various services, such as a backend management server (for example only) that supports requests sent by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The backend management server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated based on user requests) to the terminal devices. The process by which server 105 adjusts the caching strategy of the caching system 106 can be triggered by a scheduled task.

[0041] It should be noted that the caching method provided in this application embodiment can generally be executed by server 105. Correspondingly, the caching device provided in this application embodiment can generally be located in server 105. The caching method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, the caching system 106, and / or server 105. Correspondingly, the caching device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, the caching system 106, and / or server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, servers, and caching systems shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, servers, and caching systems can be included.

[0043] The following will be based on Figure 1 The described scene, through Figure 2 The caching method according to the embodiments of this application will be described in detail.

[0044] Figure 2 A flowchart illustrating a caching method according to an embodiment of this application is shown.

[0045] like Figure 2 As shown, the caching method in this embodiment includes operations S210 to S240.

[0046] In operation S210, based on the access logs of cached data, the characteristic elements of the cached data are determined. The characteristic elements include business tags and access distribution characteristic data that can indicate the access status of the cached data.

[0047] In operation S220, vector data points of cached data are constructed based on cached data, business tags, and access distribution characteristic data.

[0048] In operation S230, cluster analysis is performed on the vector data points to obtain clusters.

[0049] In operation S240, the caching strategy for cached data is adjusted based on the comparison results between the cluster feature data of the clusters and the preset threshold, and the cached data is cached according to the adjusted caching strategy.

[0050] In some embodiments, the above method provides an intelligent cache optimization process, which refers to the process of dynamically adjusting the caching strategy based on different data access patterns, cache data change frequency, and business needs. Through intelligent analysis methods such as machine learning algorithms, the caching system can optimize the storage and refresh strategies of the cache based on factors such as the real-time access status, importance, and update frequency of the cached data, dynamically adjust the expiration time of the cached data, and update the cache to improve the cache hit rate and ensure data consistency.

[0051] Dynamically adjusting the expiration time of cached data refers to a strategy that dynamically adjusts the expiration time of cached data based on factors such as cached data access patterns, update frequency, and business needs. Dynamically adjusting the expiration time of cached data allows for real-time adjustments to the cache's lifecycle based on actual conditions, ensuring the real-time nature of frequently accessed data and the appropriate storage of infrequently accessed data.

[0052] Cache updates refer to modifying or refreshing data in the cache based on changes. Cache updates can be full updates or incremental updates; the latter only updates the changed data, reducing resource consumption and improving system performance.

[0053] In some embodiments, the cached data described above may be a copy of the original data stored in the cache system, used to accelerate access to the original data. The original data may be data stored in a database. By storing copies of the original data, which are accessed more frequently than a predetermined value, in the cache system, the access speed of the original data can be improved. Accessing data in the cache system is faster than accessing data in the database.

[0054] In some embodiments, the access log can be a file used to reflect cache usage patterns and record operations such as reading, writing, updating, or deleting cached data, providing support for analyzing and optimizing caching strategies. The access log may include an access identifier, cache data hit status, access timestamp, a field indicating whether login is required for access, the access path, and the object identifier of the object accessing the cached data.

[0055] In some embodiments, the feature elements of cached data can be a set of indicators representing the multidimensional characteristics of the cached data, used to quantify the business attributes and access patterns of the cached data. Feature elements may include business tags and access distribution feature data. Business tags can describe business attributes, such as homepage caching, product details, and activity details. Access distribution feature data can represent the access patterns of the cached data, such as access frequency, difference in access time distribution, concentration of access periods, login dependency, and the proportion of accessed objects.

[0056] In some embodiments, for each cached data, a one-dimensional business label and five-dimensional access distribution feature data can be obtained. Based on the cached data, business label, and access distribution feature data, a six-dimensional vector data point can be constructed for each cached data. This vector data point can be used as input data for cluster analysis.

[0057] In some embodiments, clustering analysis of vector data points can be performed using a density-based unsupervised clustering algorithm. This algorithm divides multiple vector data points into multiple clusters based on their distribution density and identifies noisy vector data points that do not form clusters. Neighborhoods containing at least a predetermined number of vector data points form clusters; neighborhoods with fewer than the predetermined number of vector data points do not form clusters and are considered noisy vector data points. This clustering algorithm does not require a preset number of clusters, can handle clusters of arbitrary shapes, and is robust to noise. Multiple clusters can be obtained by performing clustering analysis on the vector data points. Each cluster may include a core point, which can be considered the cluster center.

[0058] In some embodiments, cluster feature data can be data that describes the characteristics of a cluster, such as the frequency of access to a cluster and the data size of a cluster.

[0059] In some embodiments, the preset threshold may include a preset access frequency threshold. The access frequency of clusters based on cluster feature data is compared with the preset access frequency threshold. The comparison result may include a cluster whose access frequency is greater than or equal to the preset access frequency threshold, indicating that the cluster is a high-frequency cluster; or a cluster whose access frequency is less than the preset access frequency threshold, indicating that the cluster is a low-frequency cluster.

[0060] In some embodiments, the caching strategy is adjusted based on the comparison results. The caching strategy can be the caching method for the cached data. For example, for high-frequency clusters, the caching strategy can be adjusted to extend the expiration time of high-frequency clusters, and for low-frequency clusters, the caching strategy can be adjusted to shorten the expiration time of low-frequency clusters.

[0061] In some embodiments, during the clustering analysis of vector data points, some vector data points may be marked as noisy vector data points. Noisy vector data points can be dynamically cached. For example, the number of accesses to a noise point within the most recent time window (e.g., 5 minutes) can be counted. If the number of accesses increases or exceeds a predetermined number, the cached data corresponding to the vector data point can be temporarily cached.

[0062] According to the caching method, apparatus, electronic device, and storage medium provided in the embodiments of this application, the characteristic elements of cached data are determined by access logs based on cached data; vector data points are constructed based on cached data, business tags, and access distribution characteristic data; cluster analysis is performed on the vector data points to obtain clusters; the caching strategy is adjusted based on the comparison results of the cluster characteristic data of the clusters with preset thresholds, and the cached data is cached according to the adjusted caching strategy. Because the caching process constructs vector data points of cached data based on multi-dimensional data such as cached data, business tags, and access distribution characteristic data, performs cluster analysis on the vector data points, and dynamically adjusts the caching strategy based on the analysis results, the cached data can flexibly change according to the dynamic actual access situation, optimizing the utilization rate of cache resources and achieving the technical effect of improving cache resource utilization.

[0063] In some embodiments, the access distribution characteristic data in the access log can be obtained as follows: the access log can be an access log within a predetermined time period, such as a log within one day, a log within one week, etc. The process of determining the characteristic elements of cached data based on the access log of cached data may include the following operations: determining the total number of accesses to the cached data within the predetermined time period based on the accessed identifiers in the access log, and obtaining the access frequency of the cached data based on the ratio between the total number of accesses and the predetermined time period; determining the standard deviation of the access time distribution and the concentration of access time periods of the cached data based on the access timestamps in the access log; determining the login dependency of the cached data based on the fields in the access log used to indicate access after login; determining the number of objects accessing the cached data based on the object identifiers in the access log, and obtaining the object proportion of the cached data based on the ratio between the number of objects and the total number of objects within the predetermined time period; obtaining the access distribution characteristic data based on the access frequency, standard deviation of the access time distribution, concentration of access time periods, login dependency, and object proportion of the cached data.

[0064] In some embodiments, each access to cached data can generate an accessed identifier, which may include fields indicating that the data has been accessed, identifiers, etc. The access frequency of the cached data can be determined by the ratio of the total number of times the cached data is accessed within a predetermined time period to the predetermined time period itself. Examples include the number of daily accesses or the number of weekly accesses.

[0065] In some embodiments, the standard deviation of the access time distribution can be obtained based on the access timestamp of the cached data, which is used to represent the degree of dispersion of the access time of the cached data.

[0066] In some embodiments, based on the access timestamps of cached data, the concentration of access time periods can also be obtained, such as being concentrated during weekdays or holidays.

[0067] In some embodiments, the dependency of cached data is determined based on a boolean indicator in the access log indicating whether user login is required for access. For example, cached data may depend on login, or cached data may not depend on login.

[0068] In some embodiments, during the process of an accessing object accessing cached data, the cache log can generate an object identifier for the accessing object. The coverage rate of the cached object can be obtained by comparing the number of objects accessing the cached data with the total number of objects appearing in the access log within a predetermined time period. This coverage rate represents the proportion of independent objects associated with the cached data within the predetermined time period. The objects may include at least one of users and / or intelligent robots.

[0069] Based on the access frequency, standard deviation of access time distribution, concentration of access time period, login dependency, and object proportion of cached data, access distribution characteristic data describing the access status of cached data can be obtained.

[0070] According to the embodiments of this application, by integrating data from five dimensions, access distribution feature data can be obtained, which can realize multi-dimensional quantitative characterization of cached data access patterns, transform cached access behavior into quantifiable clustering input, provide a more accurate basis for adjusting cached strategies, and at the same time, allow cached data to change according to actual access conditions.

[0071] In some embodiments, business tags in the access log can be obtained by: extracting a source field from the access log, wherein the source field can represent the request source of the cached data, and the source field includes at least one of the following: the cache hit path of the cached data, the application programming interface request source, and the page identifier; selecting a target field from the source fields according to the preset priority of the source fields; and generating a business tag according to the business tag generation strategy corresponding to the target field.

[0072] In some embodiments, a business tagging mechanism can be introduced to attach source identification information, such as "Homepage Overview", "Account Details", "Product Page", etc., to enhance the business-awareness of cluster analysis.

[0073] In some embodiments, the source field may include a cache hit path, an Application Programming Interface (API) request source, or a page identifier. If all three fields exist in an access log, the target field can be determined from these three fields based on their predefined priority information. For example, the API request source has a higher priority than the page identifier, and the page identifier has a higher priority than the cache hit path. If the cache hit path, API request source, and page identifier all exist, the information indicated by the API request source can be read first.

[0074] In some embodiments, different source fields can correspond to different business tag generation strategies. For example, for the cache hit path field, the information indicated by the cache hit path can be parsed to obtain the service business name in the cache path. For the API request source field, the information indicated by the API request source field can be matched with a predefined API-to-business mapping table to obtain the business name mapped to the information indicated by the API request source field. For the page identifier field, the information indicated by the page identifier can be matched with a predefined page identifier-to-business mapping table to obtain the business name. Based on the business tag generation strategy corresponding to the target field among the three fields—cache hit path, API request source, and page identifier—business tags can be obtained.

[0075] In some embodiments, the six dimensions of the vector data points are characterized. Before obtaining the feature data, the data for each dimension undergoes a unified normalization process. Furthermore, before obtaining the vector data points, these six dimensions can also be normalized to ensure that data of different scales can be reasonably compared within the same vector space. The caching system constructs each cached data point into a vector data point containing the aforementioned six dimensions, serving as the basic input for the clustering analysis performed by the analysis and decision-making module. This application, through this method, can transform massive, dynamic, and complex access data into high-dimensional, computable analysis samples, providing a stable and controllable support foundation for intelligent caching strategies. By normalizing the data in each dimension and introducing business tags to improve distinguishability, the clustering effect can be ensured to be controllable and reliable.

[0076] According to embodiments of this application, by introducing business tags into vector data points, the business semantics of cached data can be strengthened, enhancing the business-awareness of cluster analysis. By determining business tags based on the priority of the source field, the efficiency and accuracy of determining business tags can be improved.

[0077] In some embodiments, the vector data points obtained by the above operations can be subjected to cluster analysis to obtain clusters. Specifically, the cluster analysis process may include the following operations: traversing the vector data points, and forming a first neighborhood with each vector data point as the center and a predetermined neighborhood radius, wherein the first neighborhood includes the center vector data point and neighboring vector data points that have a neighbor relationship with the center vector data point; constructing clusters according to the relationship between the number of vector data points in the first neighborhood and the predetermined number of vector data points, wherein the number of vector data points in the first neighborhood is obtained based on the number of neighboring vector data points.

[0078] In some embodiments, a density-based clustering algorithm can be used to cluster vector data points to effectively handle noisy data, automatically determine the number of clusters, and classify data even when the number of clusters is not explicitly specified. The core parameters of the density-based clustering algorithm may include the epsilon (Eps) radius and the predetermined number of vector data points (Minimum Number of Points, MinPts) . The epsilon radius defines the neighborhood range of a vector data point. Points within a distance of less than epsilon are considered density-reachable. MinPts represents the minimum number of vector data points in the neighborhood required for a vector data point to become a core point. In some embodiments, a higher MinPts result in more effective noise avoidance.

[0079] In some embodiments, before performing clustering analysis on vector data points, the neighborhood radius Eps and the minimum number of points MinPts can be set. For example, the minimum number of points MinPts can be set based on the feature dimension, such as twice the dimension. The neighborhood radius Eps is estimated using the k-nearest neighbor distance graph method. By plotting the distance graph from each vector data point to its k-th nearest neighbor, the value at the inflection point is selected as Eps to ensure the rationality and robustness of clustering. k can be equal to MinPts.

[0080] In some embodiments, clustering analysis can be performed based on determined Eps and MinPts. During the clustering analysis, all vector data points in the cache system can be traversed, and a first neighborhood can be formed with each vector data point as the center and Eps as the radius. The first neighborhood can include the center vector data point and neighbor vector data points that have a neighbor relationship with the vector data point.

[0081] In some embodiments, the number of vector data points in the first neighborhood can be the number of neighboring vector data points. Based on the relationship between the number of vector data points in the first neighborhood and a predetermined number of vector data points (MinPts), a cluster can be constructed. Specifically, the clustering process may include the following operations: in response to the number of vector data points in the first neighborhood being not less than the predetermined number of vector data points, marking the vector data points as the first cluster centroids; constructing an initial cluster based on the first cluster centroids and the neighboring vector data points within the first neighborhood; for each neighboring vector data point in the initial cluster, if the neighboring vector data point is a second cluster centroid, adding a second neighborhood based on the second cluster centroid to the initial cluster to obtain a cluster, wherein the vector data points in the second neighborhood are density-reachable points of the first cluster centroid, and the second cluster centroid is determined when the number of vector data points in the second neighborhood is not less than the predetermined number of vector data points.

[0082] In some embodiments, if the number of vector data points in the first neighborhood is not less than a predetermined number of vector data points (MinPts), the center vector data point of the first neighborhood can be used as the core point and marked as the cluster center point. Based on the center vector data point in the first neighborhood and all the neighbor vector data points, an initial cluster can be obtained.

[0083] In the initial cluster, for each neighbor vector data point, the number of neighbor vector data points in the second neighborhood when the neighbor vector data point is used as the center vector data point can be determined. If the number of neighbor vector data points in the second neighborhood is not less than the predetermined number of vector points (MinPts), then the neighbor vector data points in this initial cluster can be used as the second cluster center point, and the number of neighbor vector data points in the second neighborhood can be used as the density reachable points of the first cluster center point. In this case, the neighbor vector nodes of the second cluster center point in the second neighborhood can be added to the initial cluster. If the number of neighbor vector data points in the second neighborhood is less than the predetermined number of vector points (MinPts), then the next neighbor vector data point can be directly traversed.

[0084] In some embodiments, density reachability points can be further expanded. For example, the number of neighboring vector data points in the third neighborhood when any density reachability point is used as the center vector data point can be determined. If the number of neighboring vector data points in the third neighborhood is not less than a predetermined number of vector points (MinPts), then the density reachability point can be used as the third cluster center. The neighboring vector data points in the third neighborhood can be density reachable points of the first cluster center. In this case, the neighboring vector nodes of the third cluster center in the third neighborhood can be added to the initial cluster. This process continues until all neighboring vector data points in the initial cluster have been traversed, or the density reachability points have been traversed, or no density reachable point of the first cluster center can be found, resulting in a cluster.

[0085] According to embodiments of this application, by performing cluster analysis on cached data, the caching system can accurately identify frequently accessed and periodically accessed data, thereby optimizing the retention strategy for cached content. Frequently accessed data can be retained in the cache for a longer period, reducing database access and improving system response speed.

[0086] In some embodiments, for example, the following three cached data entries exist:

[0087] 1. Account homepage caching, with high access frequency, concentrated distribution, and visible after login.

[0088] 2. Access to financial product details is unstable, concentrated at the beginning of the month, and has low user coverage.

[0089] 3. The event details page has sparse traffic and is only triggered on holidays.

[0090] During the clustering process, the first data point can be classified as a high-frequency core class, the second data point can be classified as a mid-frequency hotspot class, and the third data point may be marked as a noise point.

[0091] Although the density-based clustering algorithm described above does not explicitly generate cluster centers, the core points can be considered as density centers of the clusters. In this density-based clustering algorithm, Eps can be used to determine whether vector data points are close to each other, and MinPts can be used to define the density threshold. The combination of Eps and MinPts can ensure that clustering remains effective for data with different distributions.

[0092] In some embodiments, the actual clustering process may suffer from problems such as inconsistent feature scales leading to clustering distortion; large differences in density between different clusters leading to unbalanced clustering; and a high proportion of noise points affecting the stability of the results. To address these issues, this application normalizes the data across six dimensions before obtaining the vector data points to eliminate the impact of numerical scale differences on clustering; introduces business labels to improve the distinguishability of cached data; and incorporates cache domain knowledge to optimize Eps and MinPts parameters, setting different parameters for different business scenarios. Furthermore, an adaptive parameter estimation mechanism is embedded during the clustering process to automatically adjust parameters based on data distribution. For example, Eps is dynamically calculated using the k-nearest neighbor distance graph curve method, or MinPts is optimized based on cluster size feedback to ensure controllable and reliable clustering results.

[0093] In some embodiments, for the clusters obtained by the above operations, the cluster feature data of the clusters can be compared with a preset threshold, the caching strategy of the cached data can be adjusted according to the comparison result, and the cached data can be cached according to the adjusted caching strategy. Specifically, the preset threshold may include a first access rate value, a second access rate threshold, and a third access rate threshold, wherein the first access rate threshold may be greater than the second access rate threshold, and the second access rate threshold may be greater than the third access rate threshold.

[0094] The process of adjusting the caching strategy based on the comparison results may include the following operations: in response to cluster feature data being greater than or equal to a first access rate threshold, extending the expiration time of cached data within the cluster and preloading the cached data; in response to cluster feature data being greater than or equal to a second access rate threshold and less than the first access rate threshold, periodically refreshing the cached data within the cluster; in response to cluster feature data being greater than or equal to a third access rate threshold and less than the second access rate threshold, shortening the expiration time of cached data within the cluster; in response to cluster feature data being less than the third access rate threshold, dynamically caching the cached data based on the relationship between the number of accesses to the cached data within the cluster in a predetermined time window and a predetermined number of accesses.

[0095] In some embodiments, after clustering is completed, the clusters can be labeled and a differentiated caching strategy can be formulated for each type of cached data.

[0096] For example, if the cluster feature data is greater than or equal to the first access rate threshold, a high-frequency access label is assigned to the cluster. For high-frequency access clusters, the expiration time of cached data in the high-frequency access cluster can be extended, and preloading can be performed to improve the access speed of cached data in the high-frequency access cluster.

[0097] For example, if the cluster feature data is greater than or equal to the second access rate threshold and less than the first access rate threshold, a mid-frequency access label is assigned to the cluster. For cached data in the mid-frequency access cluster, it can be refreshed periodically.

[0098] For example, if the cluster feature data is greater than or equal to the third access rate threshold and less than the second access rate threshold, the cluster is assigned a cold data label. For cached data within cold data clusters, the expiration time of cached data within high-frequency access clusters can be shortened.

[0099] For example, if the cluster feature data is less than the third access rate threshold, it is a vector data point that has not formed a cluster, or a cluster composed of vector data points that are not included in the high-frequency access cluster, mid-frequency access cluster, or cold data cluster. This is labeled as noisy data, which may include the noise data marked during the clustering analysis. For noisy data, the cached data can be dynamically cached based on the relationship between the number of accesses within a predetermined time window and a predetermined number of accesses. For example, if the noisy data is accessed 8 times within a predetermined time window (e.g., 1 minute), which is greater than the predetermined number (5 times), then the noisy data can continue to be cached. If it is accessed 1 time within a predetermined time window (e.g., 1 minute), which is less than the predetermined number (5 times), then caching is not necessary. The duration of the time window and the predetermined number of accesses can be adaptively adjusted according to actual needs.

[0100] According to embodiments of this application, after clustering is completed, cluster feature data is compared with a preset threshold, and the expiration time is dynamically adjusted based on the actual data access situation. Low-frequency access data expires quickly, avoiding waste of memory resources. For high-frequency data, the cache time is extended to improve cache utilization and ensure stability and fast response under high concurrency. This enables the caching system to adaptively adjust according to real-time access data, support changes in business needs and high-concurrency scenarios, ensure that the caching strategy always matches the actual access pattern, reduce database load, and improve system scalability.

[0101] In some embodiments, the above method may further include the following operations: when the changing trend of the access distribution feature data is detected to meet the predetermined clustering reconstruction conditions, updating the vector data points to obtain updated vector data points; performing clustering analysis based on the updated vector data points to obtain multiple updated clusters; and caching the cached data according to the comparison result between the updated cluster feature data of the updated clusters and a preset threshold.

[0102] In some embodiments, the cache may have a periodic evaluation mechanism to continuously collect indicators such as cache hit rate, response latency, and access volume changes, and continuously update access distribution characteristic data. Once the access distribution characteristic data shows a significant trend and meets the predetermined clustering reconstruction conditions (e.g., the mean or standard deviation of the access distribution characteristic data is greater than a predetermined value), the cache system can automatically trigger the clustering reconstruction process.

[0103] The clustering reconstruction process may include automatically updating vector data points based on updated access distribution characteristic data to obtain updated vector data points. The updated vector data points are then clustered using the aforementioned clustering analysis process to obtain multiple updated clusters. Based on the comparison results between the cluster characteristic data of the updated clusters and the aforementioned preset threshold, the caching strategy is adjusted, and the adjusted caching strategy is used to cache the cached data.

[0104] For example, within month A, if the traffic growth of cached data C (originally labeled as medium-frequency access) for business B exceeds the predetermined growth value, after the caching system identifies the changing trend of the access distribution characteristics of cached data C, it can reclassify cached data C from "medium-frequency access" to "high-frequency access" and adjust the caching strategy of cached data C to resident caching to improve the response performance of cached data and save resources from repeated loading.

[0105] According to the embodiments of this application, by continuously collecting indicators such as the hit rate, response latency, and changes in access volume of cached data, and continuously updating access distribution characteristic data, once a significant trend in the change of access distribution characteristic data is detected, a clustering and reconstruction process is automatically triggered, which can realize dynamic caching of cached data, improve the cache data hit rate, and optimize the utilization of cache resources.

[0106] Figure 3 An architectural diagram of a caching system according to an embodiment of this application is illustrated.

[0107] like Figure 3 As shown, this application provides a caching system capable of implementing the above-described caching method.

[0108] like Figure 3 As shown, the caching system may include a system detection module 310, an analysis and decision-making module 320, and an expiration update module 330.

[0109] The system detection module 310 is responsible for real-time dynamic monitoring of cache data usage. This includes monitoring access logs for key metrics such as cache hit rate, memory usage, and database query frequency. By collecting and organizing this data, the system detection module provides accurate cache usage data, which may include access frequency, data access periods, and access hotspots. This cache usage data or access logs can then be transmitted to the analysis and decision-making module 320, providing a basis for its decision-making.

[0110] After receiving cache usage data (such as access time, frequency, path, login required, and user distribution) from the system detection module 310, the analysis and decision module 320 can process the cache usage data into structured feature data suitable for cluster analysis. The analysis and decision module 320 can use density-based clustering algorithms to analyze data access patterns. Based on factors such as access frequency and business requirements, the analysis and decision module 320 makes intelligent decisions regarding cache strategies, including which cached data should be retained, which should be evicted, and the cache expiration time. These cache strategies can be sent to the expiration update module 330. This decision-making process considers the actual data access situation and business priorities, ensuring that the cached data is always the most valuable and preventing expired data from consuming memory resources. The analysis and decision module 320 can be used to execute operations S210 to S240.

[0111] The expiration update module 330 can execute actual cache update operations based on the caching strategy provided by the analysis and decision module 320. The expiration update module 330 can, according to the instructions from the decision module, dynamically expire (which means dynamically adjusting the expiration time of cached data), update cached data (which means updating cached data), or evict expired cached data (which means deleting cached data). In this way, the expiration update module 330 can ensure that the cache always remains efficient and accurate, avoiding resource waste and performance bottlenecks under a fixed strategy.

[0112] According to the embodiments of this application, the above-described caching system combines clustering analysis and dynamic cache management, effectively addressing the shortcomings of existing caching systems and achieving the following effects: Improved cache hit rate and system performance. By performing clustering analysis on cached data, the system can accurately identify high-frequency and time-dependent access data, thereby optimizing the retention strategy for cached content. Frequently accessed data can be retained in the cache for a longer period, reducing database access and improving system response speed. Dynamically adjusted caching strategy saves memory resources. The system dynamically adjusts the cache expiration time based on data access patterns, allowing low-frequency access data to expire quickly, avoiding wasted memory resources. For high-frequency data, the cache time is extended, improving cache utilization and ensuring stability and fast response under high concurrency. Adaptive optimization adapts to business changes. The system can adaptively adjust based on real-time access data, supporting changes in business requirements and high-concurrency scenarios, ensuring that the caching strategy always matches the actual access pattern, reducing database load, and improving system scalability. The caching system of this application not only improves the access speed of cached data in high real-time and high-concurrency scenarios but also effectively improves the performance and user experience of the caching system, reducing development and maintenance costs.

[0113] It should be noted that, unless it is explicitly stated that there is a sequential order of execution between different operations, or that there is a sequential order of execution between different operations in terms of technical implementation, the execution order between multiple operations may not be significant, and multiple operations may be executed simultaneously.

[0114] Based on the above caching method, this application also provides a caching device. The following will combine... Figure 4 The device is described in detail.

[0115] Figure 4 A schematic block diagram of a caching device according to an embodiment of this application is shown.

[0116] like Figure 4 As shown, the caching device 400 of this embodiment includes a determination module 410, a construction module 420, a first clustering module 430, and a first caching module 440.

[0117] The determination module 410 is used to determine the characteristic elements of the cached data based on the access logs of the cached data. The characteristic elements include business tags and access distribution characteristic data that can represent the access status of the cached data.

[0118] Module 420 is used to construct vector data points of cached data based on cached data, business tags, and access distribution feature data.

[0119] The first clustering module 430 is used to perform cluster analysis on vector data points to obtain clusters.

[0120] The first caching module 440 is used to adjust the caching strategy of the cached data according to the comparison result of the cluster feature data of the cluster and the preset threshold, and to cache the cached data according to the adjusted caching strategy.

[0121] According to an embodiment of this application, the first clustering module 430 may include constituent units and building units.

[0122] The constituent unit is used to traverse vector data points and form a first neighborhood with each vector data point as the center and a predetermined neighborhood radius. The first neighborhood includes the center vector data point as the center and the neighboring vector data points that have a neighbor relationship with the center vector data point.

[0123] The construction unit is used to construct a cluster based on the relationship between the number of vector data points in the first neighborhood and a predetermined number of vector data points, wherein the number of vector data points in the first neighborhood is obtained based on the number of neighboring vector data points.

[0124] According to embodiments of this application, the building unit may include a tagging subunit, a building subunit, and an adding subunit.

[0125] The marking subunit is used to mark the center vector data point as the first cluster center point in response to the number of vector data points in the first neighborhood being no less than a predetermined number of vector data points.

[0126] Construct sub-units to build initial clusters based on the first cluster center and the neighbor vector data points in the first neighborhood.

[0127] Add sub-units to each neighbor vector data point of the initial cluster, and if the neighbor vector data point is the second cluster center, add the second neighborhood based on the second cluster center to the initial cluster to obtain the cluster. The vector data points in the second neighborhood are the density reachable points of the first cluster center. The second cluster center is determined when the number of vector data points in the second neighborhood is not less than a predetermined number of vector data points.

[0128] According to an embodiment of this application, the first cache module 440 may include a first adjustment unit, a second adjustment unit, a third adjustment unit, and a fourth adjustment unit.

[0129] The first adjustment unit is used to extend the expiration time of cached data within the cluster and preload the cached data in response to the cluster feature data being greater than or equal to the first access rate threshold.

[0130] The second adjustment unit is used to periodically refresh the cached data within the cluster in response to the cluster feature data being greater than or equal to the second access rate threshold and less than the first access rate threshold.

[0131] The third adjustment unit is used to shorten the expiration time of cached data within a cluster in response to cluster feature data being greater than or equal to a third access rate threshold and less than a second access rate threshold.

[0132] The fourth adjustment unit is used to dynamically cache the cached data in response to the cluster feature data being less than the third access rate threshold, based on the relationship between the number of accesses of the cached data within the cluster in a predetermined time window and a predetermined number of accesses.

[0133] According to embodiments of this application, the determining module 410 may include a first determining unit, a second determining unit, a third determining unit, a fourth determining unit, and a result unit.

[0134] The first determining unit is used to determine the total number of accesses to cached data within a predetermined time period based on the accessed identifiers in the access log, and to obtain the access frequency of cached data based on the ratio between the total number of accesses and the predetermined time period.

[0135] The second determining unit is used to determine the standard deviation of the access time distribution and the concentration of access time periods of the cached data based on the access timestamps in the access logs.

[0136] The third determining unit is used to determine the login dependency of cached data based on the fields in the access log that indicate access after login.

[0137] The fourth determining unit is used to determine the number of objects accessing cached data based on the object identifiers in the access log, and to obtain the object proportion of cached data based on the ratio between the number of objects and the total number of objects within a predetermined time period.

[0138] The results unit is used to obtain access distribution characteristic data based on the access frequency, standard deviation of access time distribution, concentration of access time period, login dependency, and object proportion of cached data.

[0139] According to embodiments of this application, the determining module 410 may further include an extraction unit, a selection unit, and a generation unit.

[0140] The extraction unit is used to extract the source field from the access log, wherein the source field can indicate the request source of the cached data, and the source field includes at least one of the following: the cache hit path of the cached data, the application programming interface request source, and the page identifier.

[0141] The selection cell is used to select the target field from the source fields based on the preset priority of the source fields.

[0142] The generation unit is used to generate business tags according to the business tag generation strategy corresponding to the target field.

[0143] According to embodiments of this application, the caching device may further include an update module, a second clustering module, and a second caching module.

[0144] The update module is used to update the vector data points when the detected trend of the access distribution feature data meets the predetermined clustering reconstruction conditions, thus obtaining the updated vector data points.

[0145] The second clustering module is used to perform clustering analysis based on the updated vector data points to obtain multiple updated clusters.

[0146] The second caching module is used to cache the cached data based on the comparison results between the updated cluster feature data of the updated cluster and the preset threshold.

[0147] According to embodiments of this application, any plurality of modules among the determining module 410, the building module 420, the first clustering module 430, and the first cache module 440 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this application, at least one of the determining module 410, the building module 420, the first clustering module 430, and the first cache module 440 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the determining module 410, the building module 420, the first clustering module 430, and the first caching module 440 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0148] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a caching method according to an embodiment of this application.

[0149] like Figure 5 As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0150] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0151] According to embodiments of this application, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0152] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0153] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 502 and / or RAM 503 and / or one or more memories other than ROM 502 and RAM 503 described above.

[0154] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the caching method provided in the embodiments of this application.

[0155] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0156] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0157] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0158] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0159] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0160] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A caching method, characterized in that, The method includes: Based on the access logs of the cached data, the characteristic elements of the cached data are determined, wherein the characteristic elements include business tags and access distribution characteristic data that can represent the access status of the cached data; Based on the cached data, the business tags, and the access distribution feature data, construct vector data points for the cached data; Cluster analysis is performed on the vector data points to obtain clusters; Based on the comparison results between the cluster feature data of the cluster and the preset threshold, the caching strategy of the cached data is adjusted, and the cached data is cached according to the adjusted caching strategy.

2. The method according to claim 1, characterized in that, The clustering analysis of the vector data points to obtain clusters includes: The vector data points are traversed, and a first neighborhood is formed with each vector data point as the center and a predetermined neighborhood radius. The first neighborhood includes the center vector data point and the neighbor vector data points that have a neighbor relationship with the center vector data point. The cluster is constructed based on the relationship between the number of vector data points in the first neighborhood and the predetermined number of vector data points, wherein the number of vector data points in the first neighborhood is obtained based on the number of neighboring vector data points.

3. The method according to claim 2, characterized in that, The step of constructing the cluster based on the relationship between the number of vector data points in the first neighborhood and the predetermined number of vector data points includes: In response to the fact that the number of vector data points in the first neighborhood is not less than the predetermined number of vector data points, the center vector data point is marked as the first cluster center point; Based on the first cluster center and the neighbor vector data points in the first neighborhood, an initial cluster is constructed; For each neighbor vector data point of the initial cluster, if the neighbor vector data point is the second cluster center, a second neighborhood composed of the second cluster center is added to the initial cluster to obtain the cluster. The vector data points in the second neighborhood are density reachable points of the first cluster center. The second cluster center is determined when the number of vector data points in the second neighborhood is not less than the predetermined number of vector data points.

4. The method according to claim 1, characterized in that, The preset threshold includes a first access rate value, a second access rate threshold, and a third access rate threshold, wherein the first access rate threshold is greater than the second access rate threshold, and the second access rate threshold is greater than the third access rate threshold. The step of adjusting the caching strategy for the cached data based on the comparison result of the cluster feature data of the clusters with a preset threshold includes: In response to the cluster feature data being greater than or equal to the first access rate threshold, the expiration time of the cached data within the cluster is extended, and the cached data is preloaded. In response to the cluster feature data being greater than or equal to the second access rate threshold and less than the first access rate threshold, the cached data within the cluster is refreshed periodically; In response to the cluster feature data being greater than or equal to the third access rate threshold and less than the second access rate threshold, the expiration time of the cached data within the cluster is shortened. In response to the cluster feature data being less than the third access rate threshold, the cached data is dynamically cached based on the relationship between the number of accesses of the cached data within the cluster in a predetermined time window and a predetermined number of accesses.

5. The method according to claim 1, characterized in that, The access log is the access log within a predetermined time period; The access logs based on cached data determine the characteristic elements of the cached data, including: Based on the accessed identifiers in the access log, determine the total number of accesses to the cached data within the predetermined time period, and obtain the access frequency of the cached data based on the ratio between the total number of accesses and the predetermined time period; Based on the access timestamps in the access logs, determine the standard deviation of the access time distribution and the concentration of access time periods for the cached data; Based on the fields in the access logs used to indicate access after login, the login dependency of the cached data is determined; Based on the object identifiers accessing the cached data in the access log, the number of objects accessing the cached data is determined, and the object proportion of the cached data is obtained based on the ratio between the number of objects and the total number of objects in the predetermined time period. The access distribution characteristic data is obtained based on the access frequency, the standard deviation of the access time distribution, the concentration of the access time period, the login dependency, and the object proportion of the cached data.

6. The method according to claim 1, characterized in that, The access logs based on cached data determine the characteristic elements of the cached data, including: Extract the source field from the access log, wherein the source field can indicate the request source of the cached data, and the source field includes at least one of the following: the cache hit path of the cached data, the application programming interface request source, and the page identifier; Select the target field from the source fields according to the preset priority of the source fields; The business tag is generated according to the business tag generation strategy corresponding to the target field.

7. The method according to claim 1, characterized in that, The method further includes: If the changing trend of the access distribution feature data is detected to meet the predetermined clustering reconstruction conditions, the vector data points are updated to obtain updated vector data points. Cluster analysis is performed based on the updated vector data points to obtain multiple updated clusters; The cached data is cached based on the comparison result between the updated cluster feature data of the updated cluster and the preset threshold.

8. A buffer device, characterized in that, The device includes: The determination module is used to determine the characteristic elements of the cached data based on the access logs of the cached data, wherein the characteristic elements include business tags and access distribution characteristic data that can represent the access status of the cached data; The construction module is used to construct vector data points of the cached data based on the cached data, the business tags, and the access distribution feature data; The first clustering module is used to perform cluster analysis on the vector data points to obtain clusters; The first caching module is used to adjust the caching strategy of the cached data according to the comparison result of the cluster feature data of the cluster and the preset threshold, and to cache the cached data according to the adjusted caching strategy.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images