Cache data processing method and device, medium and application system

Abstract

The invention belongs to the technical field of computers, and particularly relates to a cache data processing method and device, a medium and an application system.The method comprises the steps that a write request sent by a client side is received; the write request carries first service data and a first service priority of the first service data; obtaining a first access priority score of the first service data based on the first service priority and a first access feature of the first service data; the first access feature comprises an access frequency, an access interval and an update frequency; and storing the first service data to a first partition of a cache, the cache having a plurality of partitions, different partitions corresponding to different access priority score intervals, and the first partition being a partition corresponding to the first access priority score. According to the method, the overall hit rate and response efficiency of the cache are improved, the access pressure of the database is relieved, and more stable and efficient data support is provided for a service system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and particularly relates to a cache data processing method, device, medium and application system. BACKGROUND

[0002] With the rapid development of Internet business, the server needs to cope with a large number of concurrent user requests, such as commodity information query of an e-commerce platform or dynamic message pushing of a social media. Under the traditional architecture, if each request directly accesses the underlying database, the database load pressure will significantly increase, the response delay will rise, and thus the overall service performance will be affected. In order to alleviate this problem, the server cache technology is widely used, which effectively reduces the direct access to the database by storing high-frequency access data in the memory or local disk, and improves the data retrieval efficiency. However, in the face of dynamic changes of business scenarios and increasing complexity of data access patterns, the existing cache mechanism still has obvious limitations in adaptive ability.

[0003] The existing server cache data is often mixed storage, when the cache resources tend to be saturated, the data with high business value may be eliminated together with the data with low value due to not being effectively identified, thereby causing the response delay of core business functions and affecting the system service quality. At the same time, the resource allocation does not match the real-time business load, and the problems of resource redundancy or insufficient cache of core business data occur, which restricts the overall performance of the cache.

[0004] Therefore, there is an urgent need for an adaptive cache data processing method to improve the utilization rate of cache resources. SUMMARY

[0005] In view of the above problems, the application provides a cache data processing method, device, medium and application system.

[0006] In a first aspect, the application provides a cache data processing method, which comprises:

[0007] receiving a write request sent by a client; the write request carries first business data, and the first business data has a first business priority;

[0008] obtaining a first access priority score of the first business data based on the first business priority and a first access feature of the first business data; the first access feature includes access frequency, access interval and update frequency;

[0009] storing the first business data to a first partition of a cache, the cache has multiple partitions, different partitions correspond to different access priority score intervals, and the first partition is a partition corresponding to the first access priority score.

[0010] In one possible implementation, the plurality of partitions includes partitions of a first type and partitions of a second type, wherein the access priority scores corresponding to the partitions of the first type are all higher than the access priority scores corresponding to the partitions of the second type; cached data in the partitions of the first type cannot be deleted, while cached data in the partitions of the second type can be deleted; the method further includes:

[0011] If the cache size is greater than the first threshold, cached data in the second type of partitions are deleted sequentially in order of priority score range from low to high, until the cache size is less than the second threshold, or all cached data in the second type of partitions have been deleted; the second threshold is less than the first threshold.

[0012] In one possible implementation, the method further includes:

[0013] After deleting cached data in all partitions of the second type, if the cache size is still greater than or equal to the second threshold, the load level of the cache is obtained based on the second access characteristics of the client; the second access characteristics include at least one of: access concurrency, core data percentage and cache penetration rate;

[0014] Based on the load level, the cache is expanded.

[0015] In one possible implementation, the method further includes:

[0016] Receive a read request sent by the client; the read request carries an identifier of the second service data;

[0017] Based on the identifier of the second business data, query the second business data in the cache;

[0018] If the second business data is not found in the cache, then the second business data is retrieved from the database;

[0019] The second service data is returned to the client, and a second access priority score of the second service data is obtained based on the second service priority of the second service data and the second access characteristics of the second service data.

[0020] The second business data is stored in the second partition corresponding to the second access priority score.

[0021] In one possible implementation, the method further includes:

[0022] If the cached second service data is found in the cache, the cached second service data is returned to the client, and the second access feature of the second service data is updated.

[0023] Update the corresponding second access priority score based on the second service priority of the second service data and the updated second access characteristics;

[0024] Based on the updated second access priority score, determine whether to change the partition where the second business data is stored.

[0025] In one possible implementation, the plurality of partitions includes a first partition, a second partition, a third partition, and a fourth partition with corresponding access priority scores decreasing sequentially; before returning the cached second service data to the client, the method further includes:

[0026] If the cached second business data is located in the first partition, the cached second business data is updated in real time by listening to the logs in the database used to record data changes.

[0027] If the cached second service data is located in the second partition, then the cached second service data is updated based on the update cycle corresponding to the second service data; the update cycle corresponds to the access frequency of the second service data.

[0028] If the cached second business data is located in the third or fourth partition, the cached second business data is updated according to the client's read request for the second business data and the expiration status of the second business data.

[0029] In one possible implementation, returning the cached second business data to the client includes:

[0030] If the cached second service data is located in the first partition or the second partition, then the cached second service data is returned to the client.

[0031] If the cached second business data is located in the third partition and the second business data has expired, the cached second business data is returned to the client, and the new second business data is queried from the database in an asynchronous manner to update the cache.

[0032] If the cached second service data is located in the fourth partition and the second service data has expired, then new second service data is queried from the database to update the cache, and the updated second service data is returned to the client.

[0033] Secondly, this application provides an application system, including:

[0034] The client is used to send access requests for business data to the server; the access requests include read requests and write requests.

[0035] The server-side component is used to invoke the cached data grading module to process the access request.

[0036] The cache has multiple partitions, each corresponding to a different access priority score range, and is used to cache at least a portion of the data of the application system.

[0037] A database is used to persistently store the data of the application system;

[0038] The cached data classification module is used to perform cached data processing using the cached data processing method described in any one of the first aspects, based on the calls made by the server.

[0039] Thirdly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of the first aspects.

[0040] Fourthly, this application provides an electronic device, comprising: at least one processor and a memory; wherein,

[0041] The memory stores computer-executed instructions;

[0042] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the cached data processing method as described in any one of claims 1-7.

[0043] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps of the method as described in any of the first aspects.

[0044] The cached data processing method, device, medium, and application system provided in this application identify the static business priority of data during writing, using this as a basic weight; then, combined with real-time collected dynamic access characteristics (including access frequency, interval, and update frequency), a comprehensive quantitative score is calculated for the data. By mapping the total score to different pre-divided priority intervals, the method achieves the goal of intelligently allocating data to corresponding cache partitions. This method effectively ensures that high-business-value and high-access-frequency data is stably retained in the cache, significantly reducing the risk of core data being unexpectedly evicted due to resource contention. It improves the overall cache hit rate and response efficiency, reduces database access pressure, and provides more stable and efficient data support for business systems. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0046] Figure 1 The flowchart of the cached data processing method provided in the embodiments of this application Figure 1 ;

[0047] Figure 2 The flowchart of the cached data processing method provided in the embodiments of this application Figure 2 ;

[0048] Figure 3 The flowchart of the cached data processing method provided in the embodiments of this application Figure 3 ;

[0049] Figure 4 This is a schematic diagram of the application system structure provided in an embodiment of the present invention;

[0050] Figure 5 This is a hardware schematic diagram of an electronic device provided in an embodiment of the present invention.

[0051] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0054] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0055] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0056] Existing server-side caching mechanisms typically prioritize data based on access time, failing to effectively differentiate data's actual business value. For example, core transaction data and static log data differ significantly in business importance, yet they are not treated differently when allocating cache resources. When cache resources are scarce, if frequently accessed low-value data (such as temporary promotional pages) continues to occupy cache space, high-value data will be prematurely evicted. Once this data is removed, subsequent requests can only directly query the database, causing delays in core business responses and potentially triggering a surge in database pressure, impacting overall system stability.

[0057] Regarding cached data updates, existing strategies primarily rely on scheduled updates or passive invalidation updates, both of which have significant limitations. Scheduled updates synchronize data at fixed intervals, making it difficult to adapt to scenarios with high-frequency changes (such as product inventory in flash sales). This results in cached data lagging significantly behind the real data source during the update cycle, leading to data inconsistencies such as "products are sold out but still show as available for purchase." Passive invalidation updates, on the other hand, only trigger the update process when a user accesses expired data. During this process, the first request directly penetrates the database, causing cache penetration. In high-concurrency scenarios, such penetration can easily trigger instantaneous database pressure spikes, and may even lead to service unavailability.

[0058] Furthermore, existing caching systems' monitoring frameworks are generally limited to basic operational metric statistics, lacking the ability to correlate and analyze business scenario characteristics. The systems cannot identify dynamic changes in cache access patterns at different business stages; for example, traffic during promotional periods may reach several times the daily average, with a significant increase in the access frequency of core data. Because decisions to expand or shrink cache resources still rely on historical statistical data for lagging adjustments, they cannot predict real-time changes in business load. This often leads to insufficient cache resources and the unexpected removal of high-value data during peak traffic periods, while during off-peak periods (such as early morning hours), untimely resource reclamation results in idle and wasted cache resources.

[0059] To address the problems in existing technologies, this application provides a cached data processing method. This method identifies the static business priority of data during writing, using this priority as a basic weight. Then, it combines this with real-time collected dynamic access characteristics (including access frequency, interval, and update frequency) to comprehensively quantify and score the data. By mapping the total score to pre-divided priority intervals, the method intelligently allocates data to corresponding cache partitions. This method effectively ensures that high-business-value and high-access-frequency data is stably retained in the cache, significantly reducing the risk of core data being unexpectedly evicted due to resource contention. It improves the overall cache hit rate and response efficiency, reduces database access pressure, and provides more stable and efficient data support for business systems.

[0060] The technical solutions of this application and how they solve the aforementioned technical problems are described in detail below with specific embodiments. These specific embodiments may exist independently or in combination with each other. Identical or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0061] This embodiment provides a method for processing cached data. Figure 1 The flowchart of the cached data processing method provided in the embodiments of this application Figure 1 The method includes:

[0062] S101. Receive a write request sent by the client; the write request carries first business data and the first business priority of the first business data.

[0063] In this step, a write request refers to a request initiated by the client (such as a business system or application server) to store data in the caching system. The first business data refers to the specific data content that needs to be cached in this request. For example, it could be a user's personal information, product details, or a snapshot of an order.

[0064] The first business priority refers to the predefined level of data importance at the business level. It can be a static or semi-static label that can be determined in advance during the business design phase, reflecting the criticality of the data to the core business processes. For example, it can be defined as P0 (core transaction data), P1 (important product data), P2 (general configuration data), and P3 (auxiliary log data).

[0065] For example, when a client initiates a write request, the business priority tag for that data can be explicitly included in the request parameters. For instance, when calling a caching service API, a parameter indicating the first business priority can be added. The caching service interface directly reads and parses this parameter, using it as the value for the first business priority.

[0066] For example, instead of directly passing the priority label, the client can pass a "data key" or "business scenario identifier" that identifies the data type. The cache internally maintains a mapping rule table of business type-priority. When a write request is received, the data key is first parsed, and the internal mapping rule is queried based on its prefix to automatically determine its business type, and then mapped to the preset priority.

[0067] S102. Based on the first service priority and the first access characteristics of the first service data, obtain the first access priority score of the first service data; the first access characteristics include: access frequency, access interval and update frequency.

[0068] In this step, the first access characteristic refers to a set of indicators that reflect the dynamic access pattern of data, obtained through real-time monitoring and / or historical statistics, including but not limited to: access frequency, i.e., the total number of times the data has been accessed by read requests in a recent period (e.g., the past 10 minutes); access interval, i.e., the time difference between the current time and the last time the data was accessed; update frequency, i.e., the number of times the data itself has been modified in the data source (e.g., the database) in a recent period (e.g., the past 1 hour), resulting in the cache needing to be updated.

[0069] The first access priority score is a quantitative score (e.g., 0-100 points) calculated comprehensively, used to ultimately determine the storage location of data in the cache. It combines static business value (first business priority) and dynamic access patterns (first access characteristics).

[0070] For example, a base score and weight can be set for business priority and each access feature, and the final score can be calculated using a weighted formula. For example, the final score = business priority base score × W1 + access frequency normalized value × W2 + (1 / access interval normalized value) × W3 + (1 / update frequency normalized value) × W4; where W1~W4 are weight coefficients, and the normalized value is the process of scaling the original indicators to a uniform dimension (such as 0-1).

[0071] For example, multiple threshold intervals can be set for each evaluation dimension, and different score segments can be given according to the intervals in which the data falls, and finally the score segments of each dimension are added together.

[0072] For example, business priority P0: base score 50 points; P1: 40 points; P2: 30 points; P3: 20 points.

[0073] Access frequency (times / 10 minutes): >100 times: 25 minutes; 50~100 times: 15 minutes; <50 times: 5 minutes.

[0074] Access interval (seconds): <10s: 25 minutes; 10~60s: 15 minutes; >60s: 5 minutes.

[0075] Update frequency (times / hour): <5 times: 25 minutes; 5~20 times: 15 minutes; >20 times: 5 minutes.

[0076] The scores for the four dimensions are determined and then added together to obtain the total score.

[0077] S103. Store the first business data in the first partition of the cache. The cache has multiple partitions, and different partitions correspond to different access priority score ranges. The first partition is the partition corresponding to the first access priority score.

[0078] In this step, caching refers to a medium used for temporary storage of high-speed access data, which can be an in-memory database (such as Redis) or application local memory. Partitioning is the logical or physical division of cache space. Each partition is an independent storage area corresponding to a specific priority score range.

[0079] For example, a partition identifier can be determined based on the score range and appended as a prefix to the original data key. For instance, scores of 0-50 can be designated as P3, 50-70 as P2, 70-90 as P1, and 90-100 as P0. If a piece of data has a final score of 85, it belongs to P1. At storage time, its original key can be updated to a key with the partition identifier. When evicting data, it can be scanned and cleaned up by partition (via the key prefix).

[0080] For example, different score ranges can correspond to different, physically independent cache instances. For instance, an internal configuration mapping of score ranges and instance connection information can be maintained. When data needs to be stored, the corresponding target cache instance connection is selected based on its score, and then the data is stored. For example, partitions P0 and P1 are deployed on a high-performance, highly available dedicated cluster to prevent accidental deletion, while partitions P2 and P3 are deployed on a lower-cost shared cluster to save resources.

[0081] For example, multiple partitions include partitions of type 1 and partitions of type 2, where the access priority scores of partitions of type 1 are all higher than those of partitions of type 2; cached data in partitions of type 1 cannot be deleted, while cached data in partitions of type 2 can be deleted; the method further includes:

[0082] If the cache size is greater than the first threshold, delete the cached data in the second type partitions in order of priority score from low to high, until the cache size is less than the second threshold, or all cached data in the second type partitions have been deleted; and the second threshold is less than the first threshold.

[0083] In this example, the first threshold is the alert line for initiating cleanup, which can be set at a level where capacity pressure is high; the second threshold is the target line for completing cleanup, which can be set at a lower level with sufficient margin. The difference between the two provides a stable operating window. For example, with a total cache capacity of 10GB, the first threshold can be set to 90% (i.e., 9GB), meaning that when cache usage reaches 9GB, resources are considered strained, and the eviction process is automatically triggered. The second threshold is set to 80% (i.e., 8GB), meaning the eviction mechanism must continue until usage drops below 8GB. The difference between the two thresholds ensures that even with continuous new data writing, there is still at least 20% redundant space after eviction, effectively preventing frequent eviction-write fluctuations caused by capacity reaching its limit again in a short period.

[0084] This example demonstrates how establishing a non-deletable protected area avoids the pain point of accidentally discarding high-value core business data, thereby significantly improving the stability and responsiveness of core business operations. Furthermore, by setting dual thresholds and performing sequential discarding within low-priority partitions, it efficiently reclaims resources under capacity pressure while preventing the discarding operation itself from impacting system performance.

[0085] For example, the method also includes:

[0086] After deleting cached data in all second-type partitions, if the cached amount is still greater than or equal to the second threshold, the cache load level is obtained based on the client's second access characteristics; the second access characteristics include at least one of: access concurrency, core data percentage and cache penetration rate.

[0087] Based on the load level, the cache is expanded.

[0088] In this example, concurrent access refers to the number of cached requests processed per second. Core data percentage refers to the proportion of data belonging to the first type of partition (high priority, protected partition) among all current cached data. Cache penetration rate refers to the ratio of cache misses to the total number of requests. Load level is a rating used to quantify the current cache stress state based on the second access characteristic (e.g., it can be defined as low, medium, and high). It can correlate capacity with real-time business access patterns and performance.

[0089] For example, the collected feature values ​​can be input into a preset evaluation model or rule set to determine the current load level. High load level: (concurrency > 10000, cache penetration rate > 10%), core data percentage > 60%; Medium load level: concurrency > 5000, cache penetration rate > 5%; Low load level: does not meet the above conditions. When high load is determined: trigger emergency rapid expansion, immediately request resources, possibly expanding capacity by 50% at once to alleviate pressure as quickly as possible. When medium load is determined: trigger steady and gradual expansion, increasing capacity in batches of 20% according to plan, and continuing to observe changes in indicators after expansion.

[0090] This example can accurately and promptly ensure the stability and performance of the caching service during peak traffic periods or surges in core business, effectively preventing system crashes caused by insufficient cache resources; at the same time, it can also avoid unnecessary over-scaling in scenarios such as accumulation of non-core data, achieving an optimal balance between cost and performance.

[0091] The cached data processing method provided in this application identifies the static business priority of data during writing, using this as a basic weight; then, it combines real-time collected dynamic access characteristics (including access frequency, interval, and update frequency) to comprehensively quantify and score the data. By mapping the total score to different pre-divided priority intervals, it achieves the goal of intelligently allocating data to corresponding cache partitions. This method effectively ensures that high-business-value and high-access-frequency data is stably retained in the cache, significantly reducing the risk of core data being unexpectedly eliminated due to resource contention. It improves the overall cache hit rate and response efficiency, reduces database access pressure, and provides more stable and efficient data support for business systems.

[0092] This embodiment provides a method for processing cached data. Figure 2 The flowchart of the cached data processing method provided in the embodiments of this application Figure 2 .like Figure 2 As shown, in this embodiment... Figure 1 Based on the embodiments, the process of reading data is described in detail. The method includes:

[0093] S201. Receive a read request sent by the client; the read request carries an identifier for the second business data.

[0094] S202. Based on the identifier of the second business data, query the second business data in the cache.

[0095] S203. If the second business data is not found in the cache, then retrieve the second business data from the database.

[0096] S204. Return the second service data to the client, and obtain the second access priority score of the second service data based on the second service priority of the second service data and the second access characteristics of the second service data.

[0097] S205. Store the second service data in the second partition corresponding to the second access priority score.

[0098] It's important to note that when the requested data is not in the cache, the system not only correctly retrieves the data from the database and returns it to the client, but more importantly, it intelligently reloads the missing data into the cache. However, this data is not stored arbitrarily; rather, it dynamically calculates a priority score based on the data's business priority and current access characteristics, and stores it in a cache partition that matches its value.

[0099] The identifier for the second business data refers to a key or other form of identifier that can uniquely identify a piece of data. The client requests the corresponding data by providing this identifier. The database refers to the source system for persistently storing data, such as relational databases like MySQL and PostgreSQL, which is the final source of cached data. The second business priority is essentially the same as the first business priority in the write process; both refer to the static importance label of data at the business level.

[0100] S206. If the cached second business data is found in the cache, the cached second business data is returned to the client, and the second access characteristics of the second business data are updated.

[0101] S207. Update the corresponding second access priority score based on the second service priority of the second service data and the updated second access characteristics.

[0102] S208. Based on the updated second access priority score, determine whether to change the partition of the second business data storage.

[0103] It's important to note that a single new access action alters two key dynamic characteristics of the data: access frequency and access interval. If these characteristics aren't updated synchronously, the priority score will become invalid, failing to accurately reflect the data's current popularity and value. This could lead to subsequent resource eviction decisions being based on outdated information, potentially resulting in the incorrect removal of currently popular data.

[0104] For example, each time the second business data is matched, its second access characteristic can be updated immediately in place, and its priority score can be recalculated. The new score is then immediately compared with the score range of the partition where the data is currently located. If they do not match, a synchronous or asynchronous data migration task is initiated: the second business data is deleted from the current partition and then stored in the target partition.

[0105] For example, when the second business data is hit, only lightweight operations can be performed: directly return the second business data and record this access in a counting structure (e.g., increment a counter). Simultaneously, a periodic background task (e.g., executed every 5 minutes) is set up to scan all recorded access logs, batch update the access characteristics of the corresponding data, and recalculate the score. Finally, all data requiring partition changes is processed in batches.

[0106] The cache data processing method provided in this embodiment effectively solves the cache penetration problem. When data is not hit, it can intelligently replant it to the appropriate partition according to the calculated dynamic priority, thus transforming it into long-term cache optimization. At the same time, when data is hit, it can continuously track and update access characteristics, dynamically adjust the data's priority score and storage location, and ensure that high-value, high-frequency data is always retained in protected high-priority partitions. This systematically improves the cache hit rate, ensures the response speed of core business, and enables cache resource allocation to adapt to the dynamic changes in business traffic.

[0107] This embodiment provides a method for processing cached data. Figure 3 The flowchart of the cached data processing method provided in the embodiments of this application Figure 3 .like Figure 2 As shown, in this embodiment... Figure 4 Based on the embodiments, the caching operations prior to retrieving the cached second business data from the cache are described in detail. The method includes:

[0108] S301. Determine the partition in the cache where the second business data is located. The cache includes multiple partitions, which are the first partition, the second partition, the third partition, and the fourth partition, whose access priority scores decrease sequentially.

[0109] In this step, when the second business data is written to the cache, its priority score is matched with the partition threshold range, and the final partition identifier is associated with the second business data for storage. When the second business data expires, its partition can be determined by parsing the partition identifier of the second business data, thereby triggering the corresponding update strategy.

[0110] It should be noted that the expiration of the second business data can refer to various data failure scenarios, such as time-driven (i.e., the preset lifespan expires), event-driven (database data changes), replacement-driven (being replaced by higher priority data), and policy-driven (manual refresh).

[0111] S302. If the cached second business data is located in the first partition, the cached second business data is updated in real time by listening to the log in the database used to record data changes.

[0112] This step achieves real-time synchronization between the cache and the source data through database log monitoring, ensuring that the version of core business data (such as transaction records and inventory quantities) in the cache is always consistent with the main database, eliminating business decision risks caused by cache delays.

[0113] For example, a separate log parsing service can be deployed for the first partition to continuously monitor change events in the corresponding database tables. When a data update is detected, the parsing service immediately extracts the key-value information from the change record and proactively triggers a cache update operation: first, a new cache key is generated based on the data identifier, then the changed data is serialized and written to the cache partition, while resetting the expiration time.

[0114] S303. If the cached second business data is located in the second partition, then the cached second business data is updated based on the update cycle corresponding to the second business data; the update cycle corresponds to the access frequency of the second business data.

[0115] This step dynamically links the update cycle with access frequency, ensuring data timeliness while avoiding unnecessary real-time synchronization overhead, thus matching system resources with business needs.

[0116] For example, a sliding window counter can be maintained for the second partition's data, counting access frequency by hour. When data enters an update cycle, the scheduler dynamically calculates a new cycle value by querying the recent access count of the data and combining it with a preset threshold matrix (e.g., a cycle of 30 seconds for access counts greater than 1000, and a cycle of 2 minutes for access counts between 500 and 1000). Subsequently, a database query-cache update operation is executed synchronously, and the calculated new cycle is set as the trigger condition for the next update, forming a closed-loop control.

[0117] S304. If the cached second business data is located in the third or fourth partition, then update the cached second business data according to the client's read request for the second business data and the expiration status of the second business data.

[0118] This step achieves precise control over the consumption of low- and medium-priority cache resources by doubly binding the update action with the client's actual request and the data's expiration status, thus avoiding ineffective or premature resource allocation. For example, it will not perform updates for expired but inaccessible P3 data, nor will it trigger any operation when P4 data has not expired.

[0119] It should be noted that business data in the third or fourth partition is updated passively, not actively. Therefore, when storing this business data in the third or fourth partition, its expiration information can be included. When a client requests this business data, the expiration status of the business data is actively determined, and if expired, the corresponding update cache operation is performed.

[0120] Correspondingly, the cached second business data is returned to the client, including:

[0121] If the cached second business data is located in the first partition or the second partition, then the cached second business data is returned to the client;

[0122] If the cached second business data is located in the third partition and the second business data has expired, the cached second business data is returned to the client, and the new second business data is retrieved from the database asynchronously to update the cache.

[0123] If the cached second business data is located in the fourth partition and the second business data has expired, then query the database for new second business data to update the cache, and return the updated second business data to the client.

[0124] It should be noted that for the first partition, real-time synchronization is achieved by monitoring the database change log (such as the Binlog), ensuring that the data in the cache remains almost strongly consistent with the master database. For the second partition, periodic proactive updates are dynamically calculated based on its access frequency to keep it within an acceptable timeframe. Therefore, if the queried second business data is located in either the first or second cached partition, only the current second business data in the cache needs to be returned.

[0125] The third partition employs a strategy of returning old data and updating asynchronously. The goal is to prioritize system response speed and overall throughput when the second business data in the cache expires. Immediately returning expired cached data ensures that user requests are not blocked by time-consuming database queries, thus maintaining a smooth user experience. Simultaneously, data updates are completed asynchronously in the background, ensuring that the data eventually reaches a consistent state.

[0126] The fourth partition employs a strategy of synchronously querying and updating data, returning new data without incurring the additional overhead of asynchronous task scheduling. When the second business data in the cache expires, the database query and cache update are completed synchronously within the same request. This is a simple and resource-constrained approach that allows limited system resources to be focused on higher-priority business processes.

[0127] Correspondingly, if the cached second business data is located in the third or fourth partition, and the cached second business data has not expired, the current second business data in the cache is directly returned to the client.

[0128] This example demonstrates how differentiated data return and update strategies are implemented for partitions with different priorities, achieving a precise match between resource allocation and business value.

[0129] The cached data processing method provided in this embodiment not only provides real-time synchronization consistency guarantee for core business, but also adopts a cost-controllable update strategy for ordinary data, thereby investing the most computing resources in the most critical business data, while maintaining the simplicity of the overall architecture and improving the utilization rate of the cache.

[0130] Figure 4 This is a schematic diagram of the application system structure provided in an embodiment of the present invention. Figure 5 As shown, application system 40 includes:

[0131] Client 401 is used to send access requests for business data to the server; access requests include read requests and write requests.

[0132] A 402 error on the server side is used to invoke the cached data classification module to handle the access request;

[0133] The cache 403 has multiple partitions, with different partitions corresponding to different access priority score ranges, and is used to cache at least a portion of the data of the application system.

[0134] A database 404 error occurs when data in an application system is persistently stored.

[0135] The cached data classification module 405 is used to process cached data based on server-side calls, using any of the cached data processing methods mentioned above.

[0136] The application system provided in this embodiment of the invention interacts with the data cache by uniformly calling the cache data hierarchy module, so that the middleware can eliminate the differences caused by different systems and containers.

[0137] Figure 5 This is a hardware schematic diagram of an electronic device provided in an embodiment of the present invention. For example... Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. The device 50 also includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0138] In the specific implementation process, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above method.

[0139] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0140] In the above ​ In the illustrated embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0141] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0142] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0143] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described above.

[0144] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0145] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0146] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0147] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0148] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0149] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0150] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0151] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. The above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing cached data, characterized in that, The method includes: Receive a write request sent by the client; the write request carries first service data and a first service priority of the first service data; Based on the first service priority and the first access characteristics of the first service data, a first access priority score of the first service data is obtained; the first access characteristics include: access frequency, access interval, and update frequency. The first business data is stored in the first partition of the cache. The cache has multiple partitions, and different partitions correspond to different access priority score ranges. The first partition is the partition corresponding to the first access priority score.

2. The cached data processing method according to claim 1, characterized in that, The multiple partitions include partitions of the first type and partitions of the second type, wherein the access priority scores of the partitions of the first type are all higher than the access priority scores of the partitions of the second type. The cached data in the first type of partition cannot be deleted, while the cached data in the second type of partition can be deleted. The method further includes: If the cache size is greater than the first threshold, cached data in the second type of partitions are deleted sequentially in order of priority score range from low to high, until the cache size is less than the second threshold, or all cached data in the second type of partitions have been deleted; the second threshold is less than the first threshold.

3. The cached data processing method according to claim 2, characterized in that, The method further includes: After deleting cached data in all partitions of the second type, if the cache size is still greater than or equal to the second threshold, the load level of the cache is obtained based on the second access characteristics of the client; the second access characteristics include at least one of: access concurrency, core data percentage and cache penetration rate; Based on the load level, the cache is expanded.

4. The cached data processing method according to any one of claims 1-3, characterized in that, The method further includes: Receive a read request sent by the client; the read request carries an identifier of the second service data; Based on the identifier of the second business data, query the second business data in the cache; If the second business data is not found in the cache, then the second business data is retrieved from the database; The second service data is returned to the client, and a second access priority score of the second service data is obtained based on the second service priority of the second service data and the second access characteristics of the second service data. The second business data is stored in the second partition corresponding to the second access priority score.

5. The cached data processing method according to claim 4, characterized in that, The method further includes: If the cached second service data is found in the cache, the cached second service data is returned to the client, and the second access feature of the second service data is updated. Update the corresponding second access priority score based on the second service priority of the second service data and the updated second access characteristics; Based on the updated second access priority score, determine whether to change the partition where the second business data is stored.

6. The cached data processing method according to claim 5, characterized in that, The plurality of partitions includes a first partition, a second partition, a third partition, and a fourth partition, with access priority scores decreasing sequentially; before returning the cached second service data to the client, the method further includes: If the cached second business data is located in the first partition, the cached second business data is updated in real time by listening to the logs in the database used to record data changes. If the cached second service data is located in the second partition, then the cached second service data is updated based on the update cycle corresponding to the second service data; the update cycle corresponds to the access frequency of the second service data. If the cached second business data is located in the third or fourth partition, the cached second business data is updated according to the client's read request for the second business data and the expiration status of the second business data.

7. The cached data processing method according to claim 6, characterized in that, The step of returning the cached second service data to the client includes: If the cached second service data is located in the first partition or the second partition, then the cached second service data is returned to the client. If the cached second business data is located in the third partition and the second business data has expired, the cached second business data is returned to the client, and the new second business data is queried from the database in an asynchronous manner to update the cache. If the cached second service data is located in the fourth partition and the second service data has expired, then new second service data is queried from the database to update the cache, and the updated second service data is returned to the client.

8. An application system, characterized in that, include: The client is used to send access requests for business data to the server. The access requests include: read requests and write requests; The server-side component is used to invoke the cached data grading module to process the access request. The cache has multiple partitions, each corresponding to a different access priority score range, and is used to cache at least a portion of the data of the application system. A database is used to persistently store the data of the application system; The cached data classification module is used to perform cached data processing based on the server-side calls, using the cached data processing method described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

10. An electronic device, characterized in that, include: At least one processor and memory; wherein, The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the cached data processing method as described in any one of claims 1-7.

Images