Data caching method and apparatus, electronic device, and readable storage medium
By determining the cache level based on the operation frequency of the target data, the problem of wasted cache space in the existing technology is solved, and the cache write efficiency and the ability to dynamically change the data level are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNITED NETWORK COMM GRP CO LTD
- Filing Date
- 2023-07-18
- Publication Date
- 2026-06-26
AI Technical Summary
In existing data caching methods, data is written to various levels of cache libraries sequentially, resulting in wasted cache space resources and long write times.
By obtaining the operation frequency of the target data, the target cache level is determined, and the data is written to the corresponding cache level instead of writing to each cache level sequentially, thereby reducing the waste of cache space resources and writing time.
It reduces the waste of cache space resources, improves cache write efficiency, and enables dynamic changes to the data cache level.
Smart Images
Figure CN117009389B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data caching, and more particularly to a data caching method, apparatus, electronic device, and readable storage medium. Background Technology
[0002] To cope with the increasing volume of data access and reduce the pressure on database processing, existing applications often adopt a multi-level caching architecture to cache data, thereby improving the efficiency of data query response by setting different levels of caching.
[0003] Currently, the most common application multi-level caching is divided into three levels: application caching, shared caching, and distributed caching. When caching data, the data is written to each level of cache library in sequence, resulting in a waste of cache space resources. Summary of the Invention
[0004] This application provides a data caching method, apparatus, electronic device, and readable storage medium to solve the problem that in existing data caching operations, cached data is written sequentially to each level of cache, resulting in a waste of cache space resources.
[0005] According to a first aspect disclosed in this application, a data caching method is provided, comprising:
[0006] Obtain the target data pointed to by the current data operation; wherein, the data operation includes data write operation and data query operation;
[0007] Based on the identifier of the target data, the cache rules for the target data are queried in the cache rule statistics library; wherein, the cache rule statistics library stores the cache rules for each cached data in each level of cache, and the cache rules include the identifier of the cached data and the data cache level;
[0008] If the cache rule statistics library contains a cache rule for the target data, then the operation frequency of the target data is obtained; wherein, the operation frequency includes the data change frequency and the data query frequency;
[0009] Based on the operation frequency, determine the target cache level of the target data;
[0010] If the data operation is a data query operation and the target cache level is different from the data cache level of the target data, or if the data operation is a data write operation, then the target data is written to the cache corresponding to the target cache level;
[0011] Based on the identifier of the target data, delete the cached data corresponding to the target data from the cache corresponding to the data cache level of the target data.
[0012] In one feasible implementation, the step of acquiring the target data pointed to by the current data operation includes:
[0013] Obtain the data operation log of the current data operation; wherein, the data operation log includes the data change log of the database performing data write operations and the data query log of the application terminal performing data query operations;
[0014] The target data is obtained by parsing the data operation log.
[0015] In one feasible implementation, the frequency of acquiring the target data includes:
[0016] Collect historical operation logs within a preset time range; wherein, the historical operation logs include historical change logs of data writing operations in the database and historical query logs of data query operations in the application terminal;
[0017] The historical operation logs are parsed to obtain the number of data changes and the number of data queries for the target data;
[0018] The operation frequency of the target data is obtained based on the number of data changes, the number of data queries, and the preset time range.
[0019] In one feasible implementation, determining the target cache level of the target data based on the operation frequency includes:
[0020] The target cache level is determined based on the operation frequency and the preset cache level strategy;
[0021] The cache level strategy includes a preset mapping relationship between cache level and operation frequency.
[0022] In one feasible implementation, determining the target cache level of the target data based on the operation frequency includes:
[0023] The operation frequency is input into the cache level prediction model to obtain the target cache level output by the cache level prediction model.
[0024] Specifically, the operation frequency and data caching level of each cached data at different times are collected, and the operation frequency and data caching level of each cached data at different times are used for deep learning training to obtain the cache level prediction model.
[0025] In one feasible implementation, the target cache level includes a level 1 cache, a level 2 cache, and a level 3 cache, and the step of writing the target data into the cache corresponding to the target cache level includes:
[0026] If the target cache level is a level 1 cache, then the target data is written to the application cache corresponding to the level 1 cache;
[0027] If the target cache level is a level 2 cache, then the target data is written to the shared cache corresponding to the level 2 cache;
[0028] If the target cache level is a level 3 cache, then the target data is written to the distributed cache corresponding to the level 3 cache.
[0029] In one feasible implementation, the method further includes:
[0030] Update the data cache level in the cache rule of the target data to the target cache level.
[0031] In one feasible implementation, the caching rule further includes a cache invalidation rule corresponding to the cached data, and the method further includes:
[0032] Obtain the cache invalidation configuration rules corresponding to the target cache level;
[0033] Update the cache invalidation rule in the cache rules of the target data to the cache configuration rule.
[0034] In one feasible implementation, the method further includes:
[0035] If there is no caching rule for the target data in the caching rule statistics library, then the target cache level is determined to be a level three cache.
[0036] Write the target data into the distributed cache corresponding to the third-level cache;
[0037] The target data caching rules are generated based on the target caching level, and the target data caching rules are stored in the caching rule statistics library.
[0038] According to a second aspect disclosed in this application, a data caching device is provided, comprising:
[0039] The data acquisition module is used to acquire the target data pointed to by the current data operation; wherein, the data operation includes data query operation and data write operation;
[0040] The rule query module is used to query the cache rules of the target data in the cache rule statistics library based on the identifier of the target data; wherein, the cache rule statistics library stores the cache rules of each cached data in each level of cache, and the cache rules include the identifier of the cached data and the data cache level;
[0041] The frequency acquisition module is used to acquire the data caching level and operation frequency of the target data if the caching rule statistics library contains caching rules for the target data; wherein, the operation frequency includes data query frequency and data query change frequency;
[0042] The level determination module is used to determine the target cache level of the target data based on the operation frequency;
[0043] The cache writing module is used to write the target data into the cache corresponding to the target cache level if the data operation is a data query operation and the target cache level is different from the data cache level of the target data, or if the data operation is a data write operation.
[0044] The cache deletion module is used to delete the cached data corresponding to the target data from the cache corresponding to the data cache level of the target data, based on the identifier of the target data.
[0045] According to a third aspect disclosed in this application, an electronic device is provided, including a processor and a memory communicatively connected to the processor;
[0046] The memory stores computer-executed instructions;
[0047] The processor executes computer execution instructions stored in the memory to implement the method described in any one of the first aspects.
[0048] According to a fourth aspect disclosed in this application, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, which, when executed by a processor, are used to implement the method described in any one of the first aspects.
[0049] According to the fifth aspect disclosed in this application, a computer program product is provided, comprising a computer program that, when executed by a processor, is used to implement the method described in any one of the first aspects.
[0050] Compared with the prior art, this application has the following beneficial effects:
[0051] The data caching method, apparatus, electronic device, and readable storage medium provided in this application determine the target cache level of the target data based on the operation frequency of the target data, and then write the target data to the cache corresponding to the target cache level, instead of writing data sequentially to each level of cache as in existing technologies, thereby reducing the waste of cache space resources. Simultaneously, since only the target data is written to the cache corresponding to the target cache data, the execution time of cache writing is reduced, improving cache writing efficiency. Furthermore, the cache level of the data can be updated according to the data operation frequency, realizing dynamic changes to the data cache level. Attached Figure Description
[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without inventive effort. Wherein:
[0053] Figure 1 This application provides an illustration of an application scenario for a data caching method.
[0054] Figure 2 A flowchart illustrating a data caching method provided in an embodiment of this application;
[0055] Figure 3 A flowchart illustrating another data caching method provided in an embodiment of this application;
[0056] Figure 4 This is a schematic diagram of the structure of a data caching device provided in an embodiment of this application;
[0057] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0058] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0059] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0060] Before providing a further detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application will be explained first. The nouns and terms used in the embodiments of this application shall be interpreted as follows:
[0061] Application caching refers to a local cache used within each application, where data is stored in the application's memory. Application caching can speed up frequent data accesses and reduce the frequency of queries to the underlying data source (such as a database), thereby improving system performance. Application caching is limited to a single application and does not share cached data with other applications. Common application caching schemes include local memory caching and local file caching.
[0062] A shared cache is a cache that multiple applications can share. Data is stored on a shared cache server, providing cross-application data sharing capabilities. Multiple applications can access the same shared cache to retrieve and update data, thus avoiding redundant data loading and storage operations. Shared caches can be implemented by storing cached data on a separate cache server, or through shared memory or distributed caching technologies.
[0063] A distributed cache is a caching system that distributes cached data across multiple nodes, allowing for horizontal scaling to provide greater storage capacity and processing power. Multiple cache nodes collaborate, sharing cached data throughout the distributed system and using techniques such as consistent hashing and sharding to achieve data distribution and load balancing. Distributed caches are typically used in large-scale distributed systems to improve system scalability, fault tolerance, and performance.
[0064] With the continuous development of internet technology, data access volume is growing rapidly, putting traditional relational databases under tremendous processing pressure. Existing technologies often employ multi-level caching to reduce database pressure while improving service throughput and query response efficiency. Currently, common multi-level caching methods include application caching, shared caching, and distributed caching. Data between different levels of cache primarily relies on real-time synchronous writing. Application caching has the smallest storage space and the fastest access speed; shared caching has a larger storage space and fast access speed, but is slightly slower than application caching; distributed caching is generally built in a cluster and can store a large amount of data, but its access speed is slightly slower than the other two. The multi-level caching mechanism has the following advantages:
[0065] Firstly, it improves access speed. Different levels of caching reside in different locations, offering different access methods and speeds. Application caches typically reside within the application and are the fastest; shared caches can be shared by multiple applications and are relatively fast; while distributed caches reside on remote nodes in the network and are relatively slower. Through a multi-level caching mechanism, data can be stored in appropriate levels based on its access frequency and characteristics, thereby accelerating data reading and processing.
[0066] Secondly, multi-level caching can improve concurrency capabilities and reduce the load on upper-layer systems. Application caching, as the first-level cache closest to the application, can cache frequently used data, reducing access requirements to other caching layers and improving concurrency and response speed. Shared caching, as a public resource, allows multiple applications to share data, avoiding redundant calculations and queries, and improving overall system performance and resource utilization. Distributed caching distributes data across multiple nodes and uses distributed algorithms for data access and load balancing, supporting large-scale concurrent access and distributed computing.
[0067] Third, multi-level caching mechanisms improve reliability and availability, enhancing system fault tolerance and availability. If a cache level fails or malfunctions, the next-level cache or other backup data sources can be queried, ensuring normal system operation. Shared and distributed caches can also improve data reliability through data replication and redundant backups, avoiding single points of failure and data loss.
[0068] Fourth, optimize resource utilization. By rationally designing a multi-level caching mechanism, hardware and software resources can be allocated and utilized reasonably according to the characteristics and resource constraints of different levels. Application caching can utilize the CPU's internal registers and cache, shared caching can fully utilize the server's memory and high-speed storage media, while distributed caching can utilize the distributed storage and computing capabilities of network nodes. By optimizing resource allocation and utilization, the overall performance and efficiency of the system can be improved.
[0069] However, in existing technologies, when caching data, the data is written to various levels of cache libraries in sequence, which can lead to problems such as wasted cache space resources, long data writing execution time, and a large number of network resource interactions.
[0070] To address the aforementioned technical issues, this application determines the target cache level of the target data based on the operation frequency of the target data, thereby writing the target data into the cache corresponding to the target cache level instead of writing the data sequentially into each level of cache, thus reducing the waste of cache space resources.
[0071] Below, in conjunction with Figure 1 The application scenarios involved in the data caching method of this application are described.
[0072] Figure 1 This is a schematic diagram illustrating an application scenario of the data caching method provided in the embodiments of this application. (See attached diagram.) Figure 1 This application scenario includes a database, a data caching device, a shared cache, a distributed cache, and an application terminal, as well as an application cache within the application terminal. The application terminal contains an application program. The data caching device is connected to the database, the distributed cache, the shared cache, and the application terminal for data caching operations. In addition to connecting to its internal application cache, the application terminal is also connected to the shared cache, the distributed cache, and the database for data query operations.
[0073] Specifically, the application terminal can be an electronic device such as a smartphone, tablet computer, laptop computer, or desktop computer, and the terminal device can perform human-computer interaction through a keyboard, mouse, remote control, touchpad, or voice control device, etc. This application embodiment does not limit this.
[0074] Specifically, application caching refers to local memory within the application terminal, while shared caching, distributed caching, and data caching devices are all independent servers. These servers can be standalone physical servers, server clusters or distributed systems comprised of multiple physical servers, or even cloud servers.
[0075] It should be noted that, Figure 1 This is merely a schematic diagram illustrating an application scenario provided by an embodiment of this application. This embodiment of the application does not necessarily represent an application scenario. Figure 1 The document does not limit the actual form of the various devices included, nor does it specify the form of the devices. Figure 1 The interaction or connection methods between various devices are limited, and can be set according to actual needs in the specific application of the technical solution.
[0076] It should be noted that the data caching method provided in this application embodiment is executed by a data caching device.
[0077] In light of the above scenarios, the technical solution of the traffic information prompting method provided in this application will be described in detail below through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other; for the same or similar content, descriptions may not be repeated in different embodiments.
[0078] Figure 2 A flowchart illustrating a traffic information prompting method provided in this application embodiment is shown below. Figure 2 In some embodiments, the traffic information prompting method includes the following steps:
[0079] S201, obtain the target data pointed to by the current data operation; where the data operation includes data write operation and data query operation.
[0080] Data operations generally include two types: one is data writing operations, which are further divided into two cases: writing new data and rewriting existing data; the other is data query operations, where the target data of the data query operation is the query result returned by the query operation, and the query result is the data that has been stored in the database or cache.
[0081] Preferably, the acquisition of the target data indicated by the current data operation includes: acquiring the data operation log of the current data operation; wherein the data operation log includes the data change log of the database performing data write operations and the data query log of the application terminal performing data query operations; and parsing the data operation log to obtain the target data.
[0082] It is known that the database is responsible for writing data, while the application terminal is responsible for querying data. Both data write operations in the database and data query operations in the application terminal generate corresponding data operation logs. Specifically, these are data change logs for database write operations and data query logs for application terminal data query operations. By parsing these logs, the target data of the current data operation can be obtained. Specifically, data write operations point to newly written data, while data query operations point to data already stored in the cache or database.
[0083] Specifically, data change logs typically record information such as the type of data operation, the object of the data operation, the data before the operation, the data after the operation, the operation timestamp, and user information. Data query logs typically record information such as the data query statement, the data query result, the query execution time, the operation timestamp, user information, as well as the total application cache space, the used cache space, the identifier and type of cached data, the size of cached data, and cache retrieval success or failure indicators.
[0084] S202, based on the identifier of the target data, query the cache rules of the target data in the cache rule statistics library; wherein, the cache rule statistics library stores the cache rules of each cached data in each level of cache, and the cache rules include the identifier of the cached data and the data cache level.
[0085] The cache rule statistics library stores the cache rules for each cached data at each level of the cache. If the target data pointed to by the current data operation has already been cached, then the target data will have a corresponding cache rule in the cache rule statistics library. If there is no cache rule corresponding to the target data in the cache rule statistics library, it means that the target data has not been cached.
[0086] Specifically, querying the caching rules for target data reveals that data is stored in key-value pairs (key, value). The key identifies the data, serving as its unique identifier, allowing for data retrieval and access. The associated value is the corresponding data content. By storing data in the cache as key-value pairs, the corresponding value can be easily retrieved based on the key, improving data read / write efficiency and access speed.
[0087] Therefore, based on the characteristics of data storage, the cache rule statistics library stores cache rules including the identifier of each cached data and its corresponding cache level. Target data can then use its own identifier as a query index to check if a corresponding cache rule exists in the cache rule statistics library.
[0088] S203, if there are cache rules for the target data in the cache rule statistics library, then obtain the operation frequency of the target data; whereby the operation frequency includes the data change frequency and the data query frequency.
[0089] If the cache statistics library contains caching rules for the target data, it indicates that the target data has already been cached. In this case, by obtaining the operation frequency of the target data, it can be further determined whether the cache location of the target data needs to be changed.
[0090] In addition, cached data previously cached by the target data can also be obtained based on the target data's caching rules and the target data's identifier.
[0091] Preferably, obtaining the operation frequency of the target data includes: collecting historical operation logs within a preset time range; wherein, the historical operation logs include historical change logs of data writing operations in the database and historical query logs of data query operations in the application terminal; parsing the historical operation logs to obtain the number of data changes and the number of data queries of the target data; and obtaining the operation frequency of the target data based on the number of data changes, the number of data queries, and the preset time range.
[0092] Within a preset time frame, historical operation logs from the database and application terminals are collected. After parsing these logs, historical change and query records of the target data are retrieved based on its identifier. The number of data changes and queries for the target data are then tallied. Finally, the data change frequency and data query frequency of the target data are calculated using a frequency calculation method.
[0093] Specifically, since frequency represents the number of operations completed per unit of time, the frequency calculation method satisfies the following formula:
[0094]
[0095] Where F represents frequency, C represents number of times, and T represents time.
[0096] S204, determine the target cache level of the target data based on the operation frequency.
[0097] The basic principle for determining the target cache level of target data based on operation frequency is that the higher the operation frequency of target data, the higher the cache level should be to improve its access speed.
[0098] Preferably, determining the target cache level of the target data based on the operation frequency includes: determining the target cache level based on the operation frequency and a preset cache level strategy; wherein the cache level strategy includes a preset mapping relationship between the cache level and the operation frequency.
[0099] The target cache level of the target data can be determined by the preset mapping relationship between the operation frequency and the cache level. The specific mapping relationship can be calibrated by means of experimentation.
[0100] For example, assuming a week as the unit of time, the mapping relationship between cache level and operation frequency is shown in Table 1:
[0101] Table 1. Mapping Relationship between Cache Level and Operation Frequency
[0102]
[0103] Preferably, determining the target cache level of the target data based on the operation frequency includes: inputting the operation frequency into the cache level prediction model to obtain the target cache level output by the cache level prediction model; wherein, the operation frequency and data cache level of each cached data in different periods are collected, and the operation frequency and data cache level of each cached data in different periods are used for deep learning training to obtain the cache level prediction model.
[0104] Specifically, deep learning is used to train a cache level prediction model, which takes the operation frequency as input and the target cache level as output, by using operation frequency and data cache level as training sets. The cache level prediction model can be constructed using a Bayesian algorithm.
[0105] By employing a cache level prediction model, we can analyze the data cache usage patterns over different periods, such as daily, weekly, and monthly, and adjust the weight values of data changes and queries. For example, if it is predicted that the query volume for a certain data change will increase, the weight value will be increased based on the increasing trend, and the operation frequency will be recalculated. Thus, the target cache level can be predictively adjusted based on the changes in operation frequency in future periods.
[0106] S205, if the data operation is a data query operation and the target cache level is different from the data cache level of the target data, or if the data operation is a data write operation, then write the target data into the cache corresponding to the target cache level.
[0107] For data query operations, the operation itself does not modify the target data. If a caching rule for the target data exists in the caching rule statistics library, it indicates that the target data targeted by the data query operation is the cached data within the cache corresponding to the target data's caching level. In this case, if the target cache level is the same as the target data's cache level, no cache modification operation is needed. However, if the target cache level is different from the target data's cache level, the cache location of the target data needs to be changed; the target data needs to be rewritten to the cache corresponding to the target cache level. For example, if the original cached data was at level three and cached in the corresponding distributed cache, and now the target cache level is level two, the cached data needs to be moved to the shared cache corresponding to the level two cache.
[0108] Specifically, for data write operations, the operation indicates that the target data has been modified. In this case, regardless of whether the target cache level is the same as the target data's cache level, the newly modified target data must be written to the cache corresponding to the target cache level. Since each data write operation updates the cache, the cache data can be refreshed promptly, ensuring the accuracy of data queries.
[0109] Preferably, the current cache level includes level 1 cache, level 2 cache, and level 3 cache. Writing the target data to the cache corresponding to the current cache level includes: if the current cache level is level 1 cache, then write the target data to the application cache corresponding to level 1 cache; if the current cache level is level 2 cache, then write the target data to the shared cache corresponding to level 2 cache; if the current cache level is level 3 cache, then write the target data to the distributed cache corresponding to level 3 cache.
[0110] Each cache level has its corresponding cache. The higher the cache level, the more frequently the target data is accessed, and the faster the access speed needs to be cached. Therefore, the first-level cache corresponds to the application cache, the second-level cache corresponds to the shared cache, and the third-level cache corresponds to the distributed cache. The target data can be stored in the corresponding cache according to the target cache level.
[0111] Specifically, the data cache level is consistent with the target cache level, which also includes level 1 cache, level 2 cache, and level 3 cache. The correspondence between the data cache level and the cache is also the same as the correspondence between the target cache level and the cache.
[0112] S206, based on the identifier of the target data, delete the cached data corresponding to the target data from the cache corresponding to the data cache level of the target data.
[0113] After writing the target data to the cache corresponding to the target cache level, it is also necessary to delete the previously cached data to ensure that only the latest cached data is stored in the cache. To do this, firstly, the cached data of the previously cached data is determined based on the target data's data cache level. Then, based on the target data's identifier, the cached data corresponding to the target data is retrieved from the cache corresponding to the target data's data cache level and deleted, retaining only the target data cached in the cache corresponding to the target cache level, thus ensuring the accuracy of the data caching.
[0114] In this embodiment, when caching target data, the target cache level is determined based on the operation frequency of the target data. The target data is then written to the cache corresponding to the target cache level, instead of sequentially writing data to each cache level as in existing technologies, thus reducing the waste of cache space resources. Furthermore, since only the target data is written to the cache corresponding to the target cache data, the cache write execution time is reduced, improving cache write efficiency. Moreover, the data cache level can be updated according to the data operation frequency, enabling dynamic changes to the data cache level.
[0115] Furthermore, the data caching method can be executed asynchronously or synchronously. For asynchronous execution, a message queue can be used to retrieve data operation logs, with data change logs recorded in a data change message queue and data query logs in a data query message queue. Subsequently, by extracting and parsing messages from these queues, the target data pointed to by the data operation can be obtained. Asynchronous operations via message queues separate data operations from data caching, allowing data operations to proceed without waiting for caching to complete, thus improving application performance and concurrency capabilities while reducing latency.
[0116] For synchronous execution, when parsing messages in the message queue, the message timestamp is used to determine whether the message processing delay exceeds a preset delay threshold. If the delay exceeds the preset threshold, the target data for data operation is directly retrieved synchronously and cached.
[0117] Furthermore, for existing application data query operations, the search proceeds sequentially downwards from the application cache. If the data is not found in the application cache, the shared cache is searched; if not, the distributed cache is searched; and finally, the database is queried. If the data is found in the database, it is written to each level of cache. A large number of query requests need to traverse through various cache levels before reaching the database, resulting in numerous invalid queries to each cache level. This increases query processing latency and puts significant pressure on the database. To address this, the query frequency of the target data can be calculated during the data request process, and the cache location of the target data can be determined based on this frequency. By querying the specified cache level, the execution time of data queries is reduced.
[0118] exist Figure 2 Based on the embodiments shown, the following is combined with Figure 3 The technical solution of the above data caching method will be further introduced.
[0119] Figure 3 A flowchart illustrating another data caching method provided in this application embodiment is shown below. Figure 3 In some embodiments, the data caching method includes the following steps:
[0120] S301, obtain the target data pointed to by the current data operation; where the data operation includes data write operation and data query operation.
[0121] S302, based on the identifier of the target data, query the cache rule statistics library to see if there is a cache rule for the target data.
[0122] S303, if there are cache rules for the target data in the cache rule statistics library, then obtain the operation frequency of the target data; whereby the operation frequency includes the data change frequency and the data query frequency.
[0123] S304, determine the target cache level of the target data based on the operation frequency.
[0124] S305, determine the type of data operation.
[0125] S306, If the data operation is a data query operation, then determine whether the target cache level is the same as the data cache level of the target data.
[0126] S307 If the target cache level is different from the data cache level of the target data, then the target data is written to the cache corresponding to the target cache level.
[0127] S308, if the data operation is a data write operation, then write the target data to the cache corresponding to the target cache level.
[0128] S309, based on the identifier of the target data, delete the cached data corresponding to the target data in the cache corresponding to the data cache level of the target data.
[0129] It should be noted that the execution process of steps S301-S309 is the same as that of steps S201-S206, and will not be described again here.
[0130] S310 updates the data cache level in the target data cache rule to the target cache level.
[0131] After writing the target data into the cache corresponding to the target cache level, it is also necessary to update the data cache level in the target data caching rules according to the target cache level, and update the data cache level to the target cache level.
[0132] S311, the cache rules also include cache invalidation rules corresponding to the cached data, and obtain the cache invalidation configuration rules corresponding to the target cache level.
[0133] Existing cache invalidation rules include FIFO (First-In, First-Out), Least Recently Used (LRU), and Least Frequently Used (LFS). These require individual configuration for each application and cannot be managed uniformly for common cache categories. Different applications using the same cache may experience data errors if their configuration rules are inconsistent. Therefore, this embodiment also stores cache invalidation rules corresponding to the cached data, enabling unified management of these rules using a cache rule statistics library. After writing the target data into the cache corresponding to the target cache level, the cache invalidation rules for the target data also need to be updated. Since cache invalidation rules are determined by the cache system configuration, the invalidation configuration rules for the cache corresponding to the target cache level can be obtained.
[0134] S312, update the cache invalidation rule in the cache rules of the target data to the cache configuration rule.
[0135] Specifically, the cache invalidation rules for the target data are updated using the cache invalidation configuration rules corresponding to the target cache level.
[0136] S313 If there is no cache rule for the target data in the cache rule statistics library, then the target cache level is determined to be a level 3 cache.
[0137] If no caching rules for the target data exist in the caching rule statistics database, it indicates that the target data has not been cached before, meaning it has not been modified or queried previously. This typically corresponds to new data written to or retrieved from the database. Therefore, since this is the first time caching, the target data's target caching level can be directly set to the lowest level, i.e., level three caching.
[0138] S314: Write the target data into the distributed cache corresponding to the level 3 cache.
[0139] After determining that the target cache level of the target data is a level 3 cache, it is written into the distributed cache corresponding to the level 3 cache.
[0140] In this case, existing technologies cannot directly write data without cached records to the cache. Instead, they require a forward query to trigger an update, increasing the pressure on data queries and service execution time. This has a significant impact on systems with frequent business updates and changes. This application, however, can directly write such data to the cache, reducing the pressure on subsequent database queries and the execution time of the query service.
[0141] S315 generates caching rules for target data based on the target caching level and stores the caching rules for target data in the caching rule statistics library.
[0142] Since this is the first time the target data is cached, after caching the target data to the cache corresponding to the target cache level, caching rules for the target data are generated according to the target cache level and stored in the cache rule statistics library for subsequent data caching operations.
[0143] In this embodiment, after writing the target data into the cache corresponding to the target cache level, the method also includes updating the cache rules and cache invalidation rules, and generating related technical solutions for cache rules, thereby improving the accuracy of data caching.
[0144] Figure 4 This is a schematic diagram of the structure of a data caching device provided in an embodiment of this application. (See attached diagram.) Figure 4 The data caching device includes various functional modules for implementing the aforementioned data caching method, and any functional module can be implemented by software and / or hardware.
[0145] In some embodiments, this data caching device includes a data acquisition module 401, a rule query module 402, a frequency acquisition module 403, a level determination module 404, a cache writing module 405, and a cache deletion module 406. Wherein:
[0146] The data acquisition module 401 is used to acquire the target data pointed to by the current data operation; wherein, the data operation includes data query operation and data write operation;
[0147] The rule query module 402 is used to query the cache rules of the target data in the cache rule statistics library based on the identifier of the target data; wherein, the cache rule statistics library stores the cache rules of each cached data in each level of cache, and the cache rules include the identifier of the cached data and the data cache level;
[0148] The frequency acquisition module 403 is used to acquire the data caching level and operation frequency of the target data if there are caching rules for the target data in the caching rule statistics library; wherein, the operation frequency includes the data query frequency and the data query change frequency;
[0149] The level determination module 404 is used to determine the target cache level of the target data based on the operation frequency;
[0150] The cache writing module 405 is used to write the target data into the cache corresponding to the target cache level if the data operation is a data query operation and the target cache level is different from the data cache level of the target data, or if the data operation is a data write operation.
[0151] The cache deletion module 406 is used to delete the cache data corresponding to the target data from the cache corresponding to the data cache level of the target data, based on the identifier of the target data.
[0152] In some embodiments, the data acquisition module 401 is specifically used for:
[0153] Retrieve the data operation log of the current data operation; the data operation log includes the data change log of the database data write operation and the data query log of the application terminal data query operation;
[0154] The data operation logs are parsed to obtain the target data.
[0155] In some embodiments, the frequency acquisition module 403 is specifically used for:
[0156] Collect historical operation logs of target data within a preset time range; the historical operation logs include historical change logs of database data writing operations to target data and historical query logs of data query operations to target data by the query data terminal;
[0157] Based on historical change logs and historical query logs, obtain the number of data changes and data queries for the target data;
[0158] The operation frequency of the target data is obtained based on the number of data changes, the number of data queries, and the preset time range.
[0159] In some embodiments, the level determination module 404 is specifically used for:
[0160] Determine the target cache level based on the operation frequency and the preset cache level strategy;
[0161] The cache level strategy includes a preset mapping relationship between cache level and operation frequency.
[0162] In some embodiments, the level determination module 404 is specifically used for:
[0163] Input the operation frequency into the cache level prediction model to obtain the target cache level output by the cache level prediction model;
[0164] This involves collecting the operation frequency and data caching level of each cached data at different times, and then using deep learning to train a cache level prediction model based on the operation frequency and data caching level of each cached data at different times.
[0165] In some embodiments, the cache writing module 405 is specifically used for:
[0166] If the target cache level is level 1 cache, then the target data is written to the application cache corresponding to the level 1 cache;
[0167] If the target cache level is level two cache, then write the target data to the shared cache corresponding to the level two cache;
[0168] If the target cache level is a level 3 cache, then the target data is written to the distributed cache corresponding to the level 3 cache.
[0169] In some embodiments, the apparatus further includes a rule update module 407, which is specifically used for:
[0170] Update the data cache level in the target data's caching rules to the target cache level.
[0171] In some embodiments, the rule update module 407 is specifically used for:
[0172] Retrieve the cache invalidation configuration rules corresponding to the target cache level;
[0173] Update the cache invalidation rule in the target data's caching rules to the cache configuration rule.
[0174] In some embodiments, the cache writing module 405 is specifically used for:
[0175] If there are no caching rules for the target data in the caching rule statistics library, then the target caching level is determined to be a level three cache.
[0176] Write the target data into the distributed cache corresponding to the third-level cache;
[0177] Generate caching rules for the target data based on the target caching level, and store the caching rules for the target data in the caching rule statistics library.
[0178] The data caching device 400 provided in this application embodiment is used to execute the technical solution provided in the aforementioned data caching method embodiment. Its implementation principle and technical effect are similar to those in the aforementioned method embodiment, and will not be repeated here.
[0179] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing elements, entirely in hardware, or partially in software via processing elements and partially in hardware. For example, the cache writing module can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as program code in the device's memory, invoked and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the processor element or through software instructions.
[0180] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. (See attached diagram.) Figure 5 The electronic device 500 includes: a processor 501, and a memory 502 communicatively connected to the processor 501;
[0181] Memory 502 stores instructions executed by the computer;
[0182] The processor 501 executes computer execution instructions stored in the memory 502 to implement the aforementioned data caching method.
[0183] In the aforementioned electronic device 500, the memory 502 and the processor 501 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines, such as bus connections. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be classified as address buses, data buses, control buses, etc., but this does not mean that there is only one bus or one type of bus. The memory 502 stores computer execution instructions that implement the aforementioned data caching method, including at least one software functional module that can be stored in the memory 502 in the form of software or firmware. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502.
[0184] The memory 502 includes at least one type of readable storage medium, not limited to Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 502 stores programs, and the processor 501 executes the programs after receiving execution instructions. Furthermore, the software programs and modules within the memory 502 may also include an operating system, which may include various software components and / or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.) and can communicate with various hardware or software components to provide an operating environment for other software components.
[0185] Processor 501 can be an integrated circuit chip with signal processing capabilities. The aforementioned processor 501 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor, or processor 501 can be any conventional processor.
[0186] The electronic device 500 is used to execute the technical solution provided in the aforementioned data caching method embodiment. Its implementation principle and technical effect are similar to those in the aforementioned method embodiment, and will not be repeated here.
[0187] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned data caching method.
[0188] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0189] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the control device of a data cache device.
[0190] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solution of the aforementioned data caching method.
[0191] In the above embodiments, those skilled in the art will understand that the above method embodiments can be implemented entirely or partially by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless network, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
[0192] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0193] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A data caching method, characterized in that, include: Obtain the target data pointed to by the current data operation; wherein, the data operation includes data write operation and data query operation; Based on the identifier of the target data, the cache rules for the target data are queried in the cache rule statistics library; wherein, the cache rule statistics library stores the cache rules for each cached data in each level of cache, and the cache rules include the identifier of the cached data and the data cache level; If the cache rule statistics library contains a cache rule for the target data, then the operation frequency of the target data is obtained; wherein, the operation frequency includes the data change frequency and the data query frequency; Based on the operation frequency, determine the target cache level of the target data; If the data operation is a data query operation and the target cache level is different from the data cache level of the target data, or if the data operation is a data write operation, then the target data is written to the cache corresponding to the target cache level; Based on the identifier of the target data, delete the cache data corresponding to the target data from the cache corresponding to the data cache level of the target data; Update the data cache level in the cache rule of the target data to the target cache level; The frequency of operations for acquiring the target data includes: Collect historical operation logs within a preset time range; wherein, the historical operation logs include historical change logs of data writing operations in the database and historical query logs of data query operations in the application terminal; The historical operation logs are parsed to obtain the number of data changes and the number of data queries for the target data; The operation frequency of the target data is obtained based on the number of data changes, the number of data queries, and the preset time range; The target cache level includes a level 1 cache, a level 2 cache, and a level 3 cache. Writing the target data into the cache corresponding to the target cache level includes: If the target cache level is a level 1 cache, then the target data is written to the application cache corresponding to the level 1 cache; If the target cache level is a level 2 cache, then the target data is written to the shared cache corresponding to the level 2 cache; If the target cache level is a level 3 cache, then the target data is written to the distributed cache corresponding to the level 3 cache; The method further includes: If there is no caching rule for the target data in the caching rule statistics library, then the target caching level is determined to be a level three cache. Write the target data into the distributed cache corresponding to the third-level cache; The target data caching rules are generated based on the target caching level, and the target data caching rules are stored in the caching rule statistics library.
2. The method according to claim 1, characterized in that, The process of obtaining the target data pointed to by the current data operation includes: Obtain the data operation log of the current data operation; wherein, the data operation log includes the data change log of the database performing data write operations and the data query log of the application terminal performing data query operations; The target data is obtained by parsing the data operation log.
3. The method according to claim 1, characterized in that, Determining the target cache level of the target data based on the operation frequency includes: The target cache level is determined based on the operation frequency and the preset cache level strategy; The cache level strategy includes a preset mapping relationship between cache level and operation frequency.
4. The method according to claim 1, characterized in that, Determining the target cache level of the target data based on the operation frequency includes: The operation frequency is input into the cache level prediction model to obtain the target cache level output by the cache level prediction model. Specifically, the operation frequency and data caching level of each cached data at different times are collected, and the operation frequency and data caching level of each cached data at different times are used for deep learning training to obtain the cache level prediction model.
5. The method according to claim 1, characterized in that, Therefore, the frequency calculation method satisfies the following formula: F=C / T; Where F represents frequency, C represents number of times, and T represents time.
6. The method according to claim 1, characterized in that, The caching rules also include cache invalidation rules corresponding to the cached data, and the method further includes: Obtain the cache invalidation configuration rules corresponding to the target cache level; Update the cache invalidation rule in the cache rules of the target data to the cache configuration rule.
7. A data caching device, characterized in that, include: The data acquisition module is used to acquire the target data pointed to by the current data operation; wherein, the data operation includes data writing operation and data query operation; The rule query module is used to query the cache rules of the target data in the cache rule statistics library based on the identifier of the target data; wherein, the cache rule statistics library stores the cache rules of each cached data in each level of cache, and the cache rules include the identifier of the cached data and the data cache level; The frequency acquisition module is used to acquire the operation frequency of the target data if the cache rule statistics library contains a cache rule for the target data; wherein, the operation frequency includes the data change frequency and the data query frequency; The level determination module is used to determine the target cache level of the target data based on the operation frequency; The cache writing module is used to write the target data into the cache corresponding to the target cache level if the data operation is a data query operation and the target cache level is different from the data cache level of the target data, or if the data operation is a data write operation. The cache deletion module is used to delete the cached data corresponding to the target data in the cache corresponding to the data cache level of the target data according to the identifier of the target data; and update the data cache level in the cache rule of the target data to the target cache level; The frequency acquisition module is specifically used for: Collect historical operation logs within a preset time range; wherein, the historical operation logs include historical change logs of data writing operations in the database and historical query logs of data query operations in the application terminal; The historical operation logs are parsed to obtain the number of data changes and the number of data queries for the target data; The operation frequency of the target data is obtained based on the number of data changes, the number of data queries, and the preset time range; The cache writing module is specifically used for: If the target cache level is a level 1 cache, then the target data is written to the application cache corresponding to the level 1 cache; If the target cache level is a level 2 cache, then the target data is written to the shared cache corresponding to the level 2 cache; If the target cache level is a level 3 cache, then the target data is written to the distributed cache corresponding to the level 3 cache; If there is no caching rule for the target data in the caching rule statistics library, then the target cache level is determined to be a level three cache. Write the target data into the distributed cache corresponding to the third-level cache; The target data caching rules are generated based on the target caching level, and the target data caching rules are stored in the caching rule statistics library.
8. An electronic device, characterized in that, Includes a processor and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.