Shunting processing method, system and equipment for cache data and storage medium
By statistically stating the access frequency and configuring multi-level caching strategies, combined with the source database, the cached data memory usage problem in mobile applications is solved, the system response speed and data query efficiency are improved, and the cache usage cost is reduced.
Patent Information
- Application Number
- CN202410990577.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-26
AI Technical Summary
In the mobile application operation scenario, as the user scale and revenue activities increase, the amount of cached data continues to accumulate, and simple expiration time management is difficult to effectively solve the memory usage problem of cached data, resulting in slow system response speed and poor user experience.
By counting the access frequency of target data, determining the target path, using multi-level data cache and source database address, configuring the data expiration time, querying the data cache and source database in sequence, avoiding memory usage caused by a single cache query, and reducing cache usage cost.
It improves data query efficiency, improves system response speed, reduces memory usage, reduces cache usage costs, and avoids the problem of excessive accumulation of cached data.
Smart Images

Figure CN119025560A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of data storage technology, and in particular to a method, system, device and storage medium for offloading cache data. Background Art
[0002] At present, in the operation scenarios of mobile application activities, it is often necessary to use high-performance cache systems such as Redis (Remote Dictionary Server) to store and manage key personal information such as country codes, user profile pictures, IDs, etc. This information needs to be retained for a period of time, firstly to ensure that personal data remains unchanged for a certain period of time, and secondly to facilitate fast reading to ensure data stability and efficient reading and writing. When using Redis for caching, data is directly stored in Redis and the corresponding expiration time is set. When the expiration time is reached, the expired data will be deleted to avoid occupying cache resources.
[0003] However, as the user scale of mobile applications continues to expand and revenue activities increase, the amount of cached data is also accumulating. Simply managing cached data through expiration time is difficult to effectively solve the problem of memory usage of huge cached data, resulting in slow system response speed and poor user experience. Summary of the invention
[0004] The embodiments of the present application provide a cache data diversion processing method, system, device and storage medium, which can reduce the cache usage cost and solve the memory occupation problem of cache data while ensuring data processing efficiency.
[0005] In a first aspect, an embodiment of the present application provides a method for offloading cache data, including:
[0006] Count the access frequency of target data, which is stored in the source database;
[0007] Determine a target path of the target data from each access path based on the access frequency. Each access path corresponds to a different access frequency range setting. The access path includes addresses of several levels of data cache and source database in order of access. The data cache is used to back up the target data from the source database and configure the expiration time of the target data to delete the target data from the data cache based on the expiration time.
[0008] In response to a data access request from the requesting end for target data, the target data is obtained by sequentially querying data caches and source databases at several levels according to the target path, and the target data is returned to the requesting end.
[0009] In a second aspect, an embodiment of the present application provides a cache data offload processing system, including:
[0010] A frequency statistics module configured to count the access frequency of target data, the target data being stored in a source database;
[0011] a path determination module configured to determine a target path of target data from each access path based on an access frequency, each access path corresponding to a different access frequency range setting, the access path sequentially including addresses of a plurality of levels of data cache and a source database in an access order, the data cache being used to back up the target data from the source database, and configuring an expiration time of the target data so as to delete the target data from the data cache based on the expiration time;
[0012] The query module is configured to respond to a data access request from the requesting end for the target data, query several levels of data cache and source database in sequence according to the target path to obtain the target data, and return the target data to the requesting end.
[0013] In a third aspect, an embodiment of the present application provides a cache data offload processing device, including:
[0014] memory and one or more processors;
[0015] a memory configured to store one or more programs;
[0016] When one or more programs are executed by one or more processors, the one or more processors implement the cache data diversion processing method of the first aspect.
[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are configured to perform the cache data diversion processing method as in the first aspect.
[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the cache data diversion processing method as in the first aspect.
[0019] The embodiment of the present application counts the access frequency of the target data, and the target data is stored in the source database; based on the access frequency, the target path of the target data is determined from each access path, and each access path corresponds to a different access frequency range setting. The access path includes several levels of data cache and the address of the source database in the order of access, and the data cache is used to back up the target data from the source database, and configure the expiration time of the target data to delete the target data from the data cache based on the expiration time; in response to the data access request of the request end to the target data, the data cache and the source database of several levels are sequentially queried according to the target path to obtain the target data, and the target data is returned to the request end. By adopting the above technical means, the access path is selected according to the access frequency of the target data, and the target data is sequentially queried according to the access path, so as to avoid the memory occupation problem caused by all data being queried from the same cache, improve the query efficiency of the target data, and improve the response speed of the system. At the same time, by configuring the data expiration time, the cost of cache use can be reduced, the excessive accumulation of cache data can be avoided, and the memory occupation problem can be further reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is a flow chart of a cache data diversion processing method provided by an embodiment of the present application;
[0021] Figure 2 is a flow chart for determining the access frequency in an embodiment of the present application;
[0022] Figure 3 is a flow chart for configuring the target data expiration time in an embodiment of the present application;
[0023] Figure 4 is a flow chart of extending the expiration time of target data in an embodiment of the present application;
[0024] Figure 5 This is a schematic diagram of classification processing of cache data in an embodiment of the present application;
[0025] Figure 6 It is a structural diagram of a cache data diversion processing system provided in an embodiment of the present application;
[0026] Figure 7 It is a structural diagram of a cache data diversion processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.
[0028] A cache data diversion processing method provided in the present application aims to select an access path based on the access frequency of the target data, and query the target data in sequence according to the access path to avoid memory usage problems caused by querying all data from the same cache, thereby improving the query efficiency of the target data.
[0029] In the relevant data caching scenarios, Redis, with its excellent read and write performance, occupies a pivotal position in the caching field, providing enterprises with the ability to quickly access data and accelerate service response. When using Redis for caching, most businesses directly store data in Redis and set a corresponding expiration time (sometimes even without setting an expiration time). Although this approach is simple and direct, with the continuous expansion of the scale of enterprises and the increasing number of revenue activities, the continuous accumulation of cached data volume, the rapid growth of resource occupation, and the gradual emergence of memory occupation problems, the high memory cost has gradually become a major challenge facing enterprises. Based on this, a method for diverting and processing cache data in an embodiment of the present application is provided to solve the problem of memory occupation of cache data, while ensuring data processing efficiency, reducing the cost of cache use.
[0030] Example:
[0031] Figure 1 A flow chart of a cache data diversion processing method provided in an embodiment of the present application is given. The cache data diversion processing method provided in this embodiment can be executed by a cache data diversion processing device, which can be implemented by software and / or hardware. The cache data diversion processing device can be composed of two or more physical entities, or can be composed of one physical entity. Generally speaking, the cache data diversion processing device can be a cache data management server, a computer or other processing device.
[0032] The following description is made by taking the cache data offloading processing device as an example to perform the cache data offloading processing method. Figure 1 The cache data diversion processing method specifically includes:
[0033] S110 , counting the access frequency of target data, where the target data is stored in a source database.
[0034] When performing the diversion processing of cached data, the present application performs access frequency statistics on the target data to be queried in advance, determines the diversion processing strategy of the target data according to the statistical access frequency, and then selects the corresponding access path according to the strategy to query the target data. The target data of the present application is at least stored in the source database to provide a bottom-up strategy for the target data cache. In addition, according to the actual buffering requirements, the target data is stored in several levels of data cache, thereby solving the memory usage problem caused by a single data cache.
[0035] Specifically, the statistics of access frequency can be determined by determining the number of access requests to the target data within a period of time, and the data access frequency within the period of time can be determined by accumulating the number of access requests and dividing it by the statistical period.
[0036] Optionally, refer to Figure 2 , count the access frequency of target data, including:
[0037] S1101. Construct a primary key based on the activity identifier, data type, and statistical date of the target data;
[0038] S1102, the number of times the primary key is increased based on the data access request of the target data;
[0039] S1103: Determine the access frequency of the target data according to the primary key of the set time period.
[0040] When counting access frequency, the composite identifier of the target data's activity identifier (activity ID), data type (type), and statistical date (day) is used as the primary key (key) to track and record the access frequency of each target data to a specific data source on a specific date. Whenever the business end initiates a data access request, the system will automatically increase the count of the corresponding key. Then, the access frequency period (such as a week) is calculated according to demand, and the total number of accesses to the primary key in that period is counted. The total number of accesses divided by the period can be used to obtain the access frequency of the target data. Through long-term accumulated data access requests, the access patterns of target data of different activities in different time periods can be analyzed, so as to formulate and adjust more accurate cache data diversion strategies in the future.
[0041] S120. Determine a target path of the target data from each access path based on the access frequency. Each access path corresponds to a different access frequency range setting. The access path includes several levels of data cache and addresses of a source database in order of access. The data cache is used to back up the target data from the source database and configure an expiration time of the target data to delete the target data from the data cache based on the expiration time.
[0042] Furthermore, based on the determined access frequency, the target path of the target data can be selected from the preset multiple access paths according to the access frequency to access the target data according to the target path. Among them, the access path includes several levels of data cache and the address of the source database in the order of access, and each access path sets a corresponding number of data caches according to the actual data cache requirements. In addition, for different access frequencies of cached data, the present application sets different access paths according to the access frequency range. According to the access frequency range into which the current access frequency of the target data falls, the access path corresponding to the access frequency range is selected as the target path. Generally speaking, the higher the access frequency, the more levels of data cache its access path contains. A multi-level cache strategy for target data is implemented through multi-level data cache and a bottom-up source database, avoiding data cache pressure caused by a single cache. Data caches at different levels can back up target data from the source database in advance, and configure corresponding expiration times to regularly clean up cached data and avoid data backlogs.
[0043] For example, during the operation of application activities, activity configuration is first performed for different application activities, and then the data cache diversion strategy is configured for the activity data of each application activity. When the business party subsequently initiates an access request for the corresponding activity data, the corresponding access path is determined according to the pre-configured classification strategy and the access frequency of the activity data. The activity configuration table refers to Table 1 below:
[0044] Table 1 Activity configuration table
[0045]
[0046] During the activity configuration phase, the business side needs to accurately set the start and end time of the activity and specify the data retention period to ensure the timeliness of the final data storage. In addition, it is necessary to define the type of user personal information data required for the activity in order to configure the diversion strategy and set the access path for activity data of different types and different access frequencies.
[0047] Optionally, the data cache includes a first-level cache and a second-level cache;
[0048] The access paths include a first path, a second path, and a third path in descending order according to the access frequency range;
[0049] The first path includes the first-level cache, the second-level cache, and the source database in order of access;
[0050] The second path includes the secondary cache and the source database in the order of access;
[0051] The third path includes the source database.
[0052] The diversion strategy configuration table of this application is shown in Table 2 below:
[0053] Table 2 Traffic diversion strategy configuration table
[0054]
[0055] The diversion strategy configuration table explicitly specifies the diversion strategies applicable to active data of various data types. These strategies are divided according to levels, where the levels from low to high correspond to gradually decreasing data access frequencies. For each level, the data source access strategy and cache retention time will be set in detail. Among them, "1" in the access level is the first-level cache, and this application selects the Codis cache system as the first-level cache; "2" is the second-level cache, and this application selects the database DB as the second-level cache; "3" is the source database. For the access level "1-2-3", when accessing the active data, the access path is the first-level cache, the second-level cache, and the source database; for the access level "2-3", when accessing the active data, the access path is the second-level cache and the source database; for the access level "3", when accessing the active data, the access path is the source database.
[0056] Ultimately, these configuration data will be stored in the Apollo configuration system in a standardized JSON format to ensure that the diversion strategy can be adjusted and optimized in real time and flexibly during business activities. Based on the above diversion strategy configuration table, the data access level set for activity data of different types and different access frequencies (i.e. different levels) can be determined. Subsequently, based on the access frequency of the target data, the access level configured by its diversion strategy is determined according to the access frequency range into which the access frequency falls, so as to determine the access path of the current target data. For example, if the access frequency of the current target data is 100,000 times / 3d and the data type is "location country code", then according to the diversion strategy configuration, its access paths are the first-level cache, the second-level cache, and the source database, respectively.
[0057] By defining activity configuration and diversion strategy configuration data separately and storing them in Apollo in json format, the configuration data can be flexibly allocated during the activity cycle, so that the system can dynamically adjust the diversion strategy according to real-time needs, greatly improving the flexibility and response speed of the system. In addition, by real-time monitoring of the business's access traffic to data, the system will dynamically adjust the distribution of subsequent traffic according to the pre-defined access strategy configuration to achieve more efficient data processing and traffic management.
[0058] When the business queries the active data for the first time, the system defaults to the first-level diversion strategy for data query. At this time, since this part of the data is not yet stored synchronously in the first-level cache and the second-level cache, it is necessary to query the source database to obtain this part of the data, and return the query results to the business end in real time. At the same time, the system starts two parallel asynchronous synchronization tasks to synchronize this part of the data stored in the source database to the first-level cache and the second-level cache, and configure the expiration time of the active data in the first-level cache and the second-level cache according to the timeliness strategy configured by the diversion strategy. Subsequently, the corresponding diversion strategy can be selected according to the access frequency of this part, and then the data is accessed in sequence according to the access path until the corresponding active data is queried. By automatically selecting the most appropriate access path, the system can minimize the direct access to the source data while reducing the occupation of memory resources and improving the overall performance of the system.
[0059] Optionally, refer to Figure 3 , configure the expiration time of the target data, including:
[0060] S1201, determining the time difference between the activity end time corresponding to the target data and the current time;
[0061] S1202, determining a preset storage time limit for target data, where the storage time limit is set based on different access frequency ranges;
[0062] S1203: Select a minimum value from the time difference and the storage time limit as the expiration time of the target data.
[0063] Taking the data cache in Codis as an example, after the queried original data is stored in the Codis cache system, based on the end time of the activity corresponding to the target data, the current time and the storage time (expire) configured by the policy level, by calculating min (activity end time - current time, policy level configured expiration), take the time difference between the activity end time and the current time, and the minimum value of the storage time as the expiration time of the target data, so as to ensure the timeliness and effectiveness of the cached data and avoid the situation where the cached data is stored for a long time and causes data backlog. The database structure of the data cache is shown in Table 3 below.
[0064] Table 3 Database structure table
[0065]
[0066] The database structure table records in detail the expiration time of the queried source data cached in Codis, database DB, etc., as well as the complete expiration time of the entire data cycle. Based on the above information, the expiration time of the data cache of each target data can be determined, so that the expired target data can be deleted according to the expiration time.
[0067] When cleaning data later, the system sets a scheduled task based on the activity information configuration to scan and identify activity data that has reached the expiration date. Then, the identified expired activity data is deleted in batches, thereby effectively reducing the amount of invalid data stored in the data cache and optimizing system performance.
[0068] After configuring the expiration time of the target data, refer to Figure 4 , also includes:
[0069] S1204, determining the time difference between the storage time of the target data in the first-level cache and the expiration time;
[0070] S1205: When the time difference is lower than the set time threshold and the target path of the target data has not changed, extend the expiration time of the target data in the first-level cache.
[0071] For the first-level cache Codis, the system periodically scans the cached data stored in Codis by setting a scheduled task. When it is detected that the expiration time (TTL) of a cached data is less than one hour, the system will start an independent update thread to determine its latest diversion strategy based on the current access frequency of the target data, and then dynamically extend the expiration time TTL of the cached data according to the expiration time configured by the classification strategy to ensure that the data is fully utilized within the validity period and optimize the data caching effect.
[0072] In addition, when the business party initiates a data access request, if the requested target data does not exist in the data cache, the system will immediately query the source database and synchronize the target data to the data cache to ensure the real-time nature of the data.
[0073] S130 , in response to a data access request from the requesting end for target data, query data caches and source databases at several levels in sequence according to the target path to obtain the target data, and return the target data to the requesting end.
[0074] Finally, based on the access path of the target data determined above, the request for access to the target data is responded to by the requesting end, and several levels of data cache and source database are queried in sequence according to the access path. After the target data is queried, it is returned to the requesting end, thereby completing the query operation of the target data.
[0075] For example, refer to Figure 5 ,After the business party initiates a data access request, the access path of the target data can be determined according to the pre-configured diversion strategy of the target data, and then the corresponding access path is selected to query the target data.
[0076] Taking the access path of "first-level cache-second-level cache-source database" as an example, the data cache (first-level cache, second-level cache) is queried in sequence according to the access path. If the target data is found in a certain cache level and has not expired, the data is directly returned to the requester. If the target data is not found in all cache levels or the data has expired, the source database is queried and the result is returned to the requester. At the same time, if it is detected that the first-level cache or the second-level cache does not store the target data, the target data is backed up from the source database to the first-level cache or the second-level cache. By storing the query results of the source database in the corresponding cache level, it is convenient for quick access to future data.
[0077] As mentioned above, by counting the access frequency of the target data, the target data is stored in the source database;
[0078] Based on the access frequency, the target path of the target data is determined from each access path. Each access path corresponds to a different access frequency range setting. The access path includes several levels of data cache and the address of the source database in the order of access. The data cache is used to back up the target data from the source database, and the expiration time of the target data is configured to delete the target data from the data cache based on the expiration time; in response to the data access request of the request end for the target data, the data cache and the source database of several levels are sequentially queried according to the target path to obtain the target data, and the target data is returned to the request end. By adopting the above technical means, the access path is selected according to the access frequency of the target data, and the target data is sequentially queried according to the access path, so as to avoid the memory occupation problem caused by all data being queried from the same cache, improve the query efficiency of the target data, and improve the response speed of the system. At the same time, by configuring the data expiration time, the cache usage cost can be reduced, the excessive accumulation of cache data can be avoided, and the memory occupation problem can be further reduced.
[0079] Based on the above embodiments, Figure 6 This is a schematic diagram of the structure of a cache data diversion processing system provided by the present application. Figure 6The cache data offloading processing system provided in this embodiment specifically includes: a frequency statistics module 21, a path determination module 22 and a query module 23.
[0080] Wherein, the frequency statistics module 21 is configured to count the access frequency of target data, and the target data is stored in the source database;
[0081] The path determination module 22 is configured to determine a target path of the target data from each access path based on the access frequency, each access path is set corresponding to a different access frequency range, the access path includes addresses of a plurality of levels of data cache and a source database in sequence according to the access order, the data cache is used to back up the target data from the source database, and an expiration time of the target data is configured to delete the target data from the data cache based on the expiration time;
[0082] The query module 23 is configured to respond to a data access request of the request end for the target data, query several levels of data cache and source database in sequence according to the target path to obtain the target data, and return the target data to the request end.
[0083] Specifically, the access frequency of target data is counted, including:
[0084] Build a primary key based on the activity ID, data type and statistical date of the target data;
[0085] The number of times the primary key is incremented based on the data access request of the target data;
[0086] Determine the access frequency of the target data based on the primary key of the set time period.
[0087] Specifically, the data cache includes a first-level cache and a second-level cache;
[0088] The access paths include a first path, a second path, and a third path in descending order according to the access frequency range;
[0089] The first path includes the first-level cache, the second-level cache, and the source database in order of access;
[0090] The second path includes the secondary cache and the source database in the order of access;
[0091] The third path includes the source database.
[0092] Specifically, configure the expiration time of the target data, including:
[0093] Determine the time difference between the activity end time corresponding to the target data and the current time;
[0094] Determine the preset storage time limit for the target data, which is set based on different access frequency ranges;
[0095] Select the minimum value from the time difference and storage time limit as the expiration time of the target data.
[0096] After configuring the expiration time of the target data, it also includes:
[0097] Determine the time difference between the storage time of the target data in the first-level cache and the expiration time;
[0098] When the time difference is lower than the set time threshold and the target path of the target data has not changed, the expiration time of the target data in the first-level cache is extended.
[0099] Specifically, after sequentially querying data caches and source databases at several levels according to the target path to obtain target data, the following steps are also included:
[0100] When it is detected that the first-level cache or the second-level cache does not store the target data, the target data is backed up from the source database to the first-level cache or the second-level cache.
[0101] As mentioned above, by counting the access frequency of the target data, the target data is stored in the source database;
[0102] Based on the access frequency, the target path of the target data is determined from each access path. Each access path corresponds to a different access frequency range setting. The access path includes several levels of data cache and the address of the source database in the order of access. The data cache is used to back up the target data from the source database, and the expiration time of the target data is configured to delete the target data from the data cache based on the expiration time; in response to the data access request of the request end for the target data, the data cache and the source database of several levels are sequentially queried according to the target path to obtain the target data, and the target data is returned to the request end. By adopting the above technical means, the access path is selected according to the access frequency of the target data, and the target data is sequentially queried according to the access path, so as to avoid the memory occupation problem caused by all data being queried from the same cache, improve the query efficiency of the target data, and improve the response speed of the system. At the same time, by configuring the data expiration time, the cache usage cost can be reduced, the excessive accumulation of cache data can be avoided, and the memory occupation problem can be further reduced.
[0103] The cache data offload processing system provided in the embodiment of the present application can be configured to execute the cache data offload processing method provided in the above embodiment, and has corresponding functions and beneficial effects.
[0104] Based on the above practical example, the present application embodiment also provides a cache data flow processing device, referring to Figure 7, the cache data diversion processing device includes: a processor 31, a memory 32, a communication module 33, an input device 34 and an output device 35. The memory, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the cache data diversion processing method described in any embodiment of the present application (for example, a frequency statistics module, a path determination module and a query module in the cache data diversion processing system). The communication module is configured to perform data transmission. The processor executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory, that is, realizes the above-mentioned cache data diversion processing method. The input device can be configured to receive input digital or character information, and generate key signal input related to the user settings and function control of the device. The output device may include a display device such as a display screen. The cache data diversion processing device provided above can be configured to execute the cache data diversion processing method provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.
[0105] On the basis of the above embodiments, the embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, wherein the computer-executable instructions are configured to execute a method for shunting cache data when executed by a computer processor, and the storage medium may be any of various types of memory devices or storage devices. Of course, the computer-executable instructions of the computer-readable storage medium provided in the embodiments of the present application are not limited to the method for shunting cache data as described above, and may also execute related operations in the method for shunting cache data provided in any embodiment of the present application.
[0106] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, and includes a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the cache data diversion processing method described in each embodiment of the present application.
Claims
1. A method for distributing cache data, characterized in that: include: Counting the access frequency of target data, where the target data is stored in a source database; Determining a target path of the target data from each access path based on the access frequency, each access path being set corresponding to a different access frequency range, the access path sequentially including a plurality of levels of data cache and an address of the source database in an access order, the data cache being used to back up the target data from the source database, and configuring an expiration time of the target data so as to delete the target data from the data cache based on the expiration time; In response to a data access request from a requesting end for the target data, the data caches at several levels and the source database are sequentially queried according to the target path to obtain the target data, and the target data is returned to the requesting end.
2. The method for distributing cache data according to claim 1, characterized in that: The data cache includes a first-level cache and a second-level cache; The access paths include a first path, a second path and a third path in descending order according to the access frequency range; The first path includes the first-level cache, the second-level cache and the source database in order of access; The second path includes the secondary cache and the source database in order of access; The third path includes the source database.
3. The method for distributing cache data according to claim 2, characterized in that: After sequentially querying the data caches at several levels and the source database according to the target path to obtain the target data, the method further includes: When it is detected that the first-level cache or the second-level cache does not store the target data, the target data is backed up from the source database to the first-level cache or the second-level cache.
4. The method for distributing cache data according to claim 2, characterized in that: After configuring the expiration time of the target data, the method further includes: Determine a time difference between a storage time of the target data in the first-level cache and the expiration time; When the time difference is lower than a set time threshold and the target path of the target data has not changed, the expiration time of the target data in the first-level cache is extended.
5. The method for distributing cache data according to any one of claims 1 to 4, characterized in that: The access frequency of the statistical target data includes: Building a primary key based on the activity identifier, data type and statistical date of the target data; Increasing the primary key a number of times based on a data access request for the target data; The access frequency of the target data is determined according to the primary key of a set time period.
6. The method for distributing cache data according to any one of claims 1 to 4, characterized in that: The configuring the expiration time of the target data includes: Determine the time difference between the activity end time corresponding to the target data and the current time; Determining a preset storage time limit for the target data, wherein the storage time limit is set based on different access frequency ranges; A minimum value is selected from the time difference and the storage time limit as the expiration time of the target data.
7. A cache data distribution processing system, characterized in that: include: A frequency statistics module configured to count the access frequency of target data stored in a source database; a path determination module, configured to determine a target path of the target data from each access path based on the access frequency, each access path being set corresponding to a different access frequency range, the access path including, in order of access, a plurality of levels of data cache and an address of the source database, the data cache being used to back up the target data from the source database, and configuring an expiration time of the target data so as to delete the target data from the data cache based on the expiration time; The query module is configured to respond to a data access request of the request end for the target data, query the data caches of several levels and the source database in sequence according to the target path to obtain the target data, and return the target data to the request end.
8. A cache data distribution processing device, characterized in that: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the cache data offloading processing method as described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a computer processor, they are configured to execute the cache data offloading processing method according to any one of claims 1 to 6.
10. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the cache data offloading processing method according to any one of claims 1 to 7.
Citation Information
Cited By
Dynamic management method and system for multi-layer data cache
CN119987685A
