Cache data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a cache data processing method and device, electronic equipment and a storage medium. Relates to the technical field of caching, is applied to a target caching system, and comprises the following steps: acquiring an access record of cached data in a preset time period according to a preset time period; based on the access information of each piece of cache data included in the target access record and the member service information of the user accessing each piece of cache data, determining the popularity of each piece of cache data, and determining hotspot data in each piece of cache data; and when the survival duration of the hotspot data is not the target survival duration corresponding to the popularity of the hotspot data, adjusting the survival duration of the hotspot data to be the target survival duration, and when the actual survival duration of the hotspot data does not reach the target survival duration, continuing to store the hotspot data, otherwise, deleting the hotspot data. The survival duration of each hotspot data is dynamically adjusted according to the popularity of the cache data, so that frequent expiration of the hotspot data is avoided, and the data access efficiency is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of caching technology, and in particular to a method, apparatus, electronic device, and storage medium for processing cached data. Background Technology

[0002] Application platforms provide services to users by accessing data in databases. Since database read speeds are much slower than memory read speeds, and the QPS (Query Per Second) of database interfaces increases with the volume of user requests, some data is typically stored in a caching system to ensure data access efficiency and prevent database overload. For example, a video platform's membership transaction system covers core businesses such as membership activation, renewal, and benefits redemption. During peak periods like Spring Festival promotions and anniversary sales, user traffic surges, and the QPS of the membership transaction system's data interfaces can reach more than 10 times the normal level. Therefore, a caching system is necessary to ensure data access efficiency and prevent database overload.

[0003] The current caching system sets the same fixed TTL (Time to Live) for all cached data. Frequently expired cached data with high usage frequency during peak access periods can cause cache breakdown, requiring frequent retrieval of this data from the database and resulting in excessive database access pressure. For example, during promotional events, a large number of concurrent user requests arrive at the membership transaction system simultaneously, causing a surge in access to cached data related to core business processes such as membership activation. This cached data frequently expires, requiring the membership transaction system to frequently access the database to retrieve relevant data, leading to a sharp increase in database access pressure. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for processing cached data, so as to dynamically manage the lifespan of cached data based on its frequency of use, thereby ensuring data access efficiency. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a method for processing cached data, applied to a target caching system; the method includes:

[0006] According to a preset time period, access records of cached data related to member services in the cache of the target caching system are obtained within a preset time period, wherein the preset time period includes the current time and / or a preset time period before the current time.

[0007] Based on the access information of each cached data included in the target access record and the membership business information of the users accessing each cached data, the popularity of each cached data is determined, and based on the popularity of each cached data, the hot data in each cached data is determined, wherein the target access record is the access record obtained in the current time period.

[0008] If the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, the survival time of the hot data is adjusted to the target survival time, wherein the target survival time of the hot data is positively correlated with the popularity of the hot data;

[0009] If the actual lifespan of the hot data does not reach the target lifespan, the hot data continues to be stored in the cache; if the actual lifespan of the hot data reaches the target lifespan, the hot data is deleted from the cache.

[0010] Secondly, embodiments of this application provide a cache data management device applied to a target cache system; the device includes:

[0011] The access record acquisition module is used to acquire access records of cached data related to member services in the cache of the target caching system within a preset time period according to a preset time cycle, wherein the preset time period includes the current time and / or a preset time period before the current time.

[0012] The hotspot data determination module is used to determine the popularity of each cached data based on the access information of each cached data included in the target access record and the membership business information of the users accessing each cached data, and to determine the hotspot data in each cached data based on the popularity of each cached data, wherein the target access record is the access record obtained in the current time period.

[0013] The survival time adjustment module is used to adjust the survival time of the hot data to the target survival time when the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, wherein the target survival time of the hot data is positively correlated with the popularity of the hot data;

[0014] The processing module is configured to continue storing the hot data in the cache when the actual lifespan of the hot data does not reach the target lifespan, and to delete the hot data from the cache when the actual lifespan of the hot data reaches the target lifespan.

[0015] Thirdly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0016] Memory, used to store computer programs;

[0017] When a processor executes a program stored in memory, it implements any of the steps described above.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the steps described above.

[0019] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the steps described above.

[0020] Beneficial effects of the embodiments in this application:

[0021] In the technical solution provided by this invention, the target caching system can dynamically determine the hot data in the cached data related to member services in the target caching system according to a preset time period, and adjust the lifespan of each hot data to a target lifespan corresponding to its popularity, thereby achieving the purpose of dynamically adjusting the lifespan of cached data based on data popularity. By extending the lifespan of cached data according to its popularity, highly popular data in the cache will not frequently expire. Thus, when an access request for hot data is received, the server can directly access the hot data from the cache without reading it from the database, reducing the data access pressure on the database. Furthermore, since hot data does not frequently expire, the target caching system does not need to frequently read and cache the hot data from the database, further reducing the data access pressure on the database. This achieves the goal of avoiding cache breakdown and database access pressure surges caused by frequent expiration of hot data, improving the hot data hit rate, and ensuring data access efficiency. Of course, implementing any product or method of this application does not necessarily require achieving all of the above advantages simultaneously. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0023] Figure 1 A flowchart illustrating a method for processing cached data provided in an embodiment of this application;

[0024] Figure 2 Another flowchart illustrating the method for processing cached data provided in this application embodiment;

[0025] Figure 3 Another flowchart illustrating the method for processing cached data provided in this application embodiment;

[0026] Figure 4 for Figure 3 A flowchart illustrating a specific implementation of step S304;

[0027] Figure 5 A schematic diagram of an adaptive caching system based on JetCache provided in an embodiment of this application;

[0028] Figure 6 A flowchart illustrating the multi-dimensional hotspot identification process provided in this application embodiment;

[0029] Figure 7 A flowchart illustrating the adaptive caching strategy process provided in this application embodiment;

[0030] Figure 8 A flowchart illustrating the dynamic allocation process of cache resources provided in this application embodiment;

[0031] Figure 9 This is a schematic diagram of the structure of a cache data processing device provided in an embodiment of this application;

[0032] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0034] To manage cached data and prevent frequently expired cached data with high usage frequency, this application provides a method, apparatus, electronic device, and storage medium for processing cached data. This method can be applied to any target caching system, such as JetCache, and is not specifically limited thereto. Furthermore, the cache of the target caching system in this application can be a local cache or a remote cache, and is not specifically limited thereto.

[0035] like Figure 1 As shown, a method for processing cached data includes:

[0036] S101: According to a preset time period, obtain the access records of cached data related to member services in the cache of the target cache system within a preset time period.

[0037] The preset time period includes the current time and / or a preset time period preceding the current time.

[0038] S102: Based on the access information of each cached data included in the target access record and the membership business information of the users accessing each cached data, determine the popularity of each cached data, and based on the popularity of each cached data, determine the hot data in each cached data.

[0039] The target access record is the access record obtained in the current time period.

[0040] S103: If the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, adjust the survival time of the hot data to the target survival time.

[0041] The target lifespan of the hot data is positively correlated with its popularity.

[0042] S104: If the actual lifespan of the hot data does not reach the target lifespan, continue to store the hot data in the cache; if the actual lifespan of the hot data reaches the target lifespan, delete the hot data from the cache.

[0043] In the technical solution provided by this invention, the target caching system can dynamically determine the hot data in the cached data related to member services in the target caching system according to a preset time period, and adjust the lifespan of each hot data to a target lifespan corresponding to its popularity, thereby achieving the purpose of dynamically adjusting the lifespan of cached data based on data popularity. By extending the lifespan of cached data according to its popularity, highly popular data in the cache will not frequently expire. Thus, when an access request for hot data is received, the server can directly access the hot data from the cache without having to read it from the database, reducing the data access pressure on the database. Furthermore, since hot data does not frequently expire, the target caching system does not need to frequently read and cache the hot data from the database, further reducing the data access pressure on the database. This achieves the goal of avoiding cache breakdown and database access pressure surges caused by frequent expiration of hot data, improving the hot data hit rate, and ensuring data access efficiency.

[0044] Once data in a database is accessed, it is likely to be accessed again in a short period of time. Therefore, the data access path can be shortened by temporarily storing the data in a faster storage medium. Based on this, data can be stored in a cache so that data can be read from the cache, thereby reducing the number of database accesses and improving data reading speed and system performance.

[0045] Furthermore, reasonable cache data management strategies can be designed based on business scenarios. Current mainstream caches, such as Redis and MySQL, mainly use fixed TTLs for the stored cache data. For example, a fixed TTL of 2 hours is set for each cached data. During periods of sudden traffic surges, a large number of data access requests flood in simultaneously, and the access volume of hot data increases sharply. Because the TTLs of hot data are fixed and their expiration times are similar, hot data expires in a concentrated manner during the traffic surge. Requests directly penetrate to the database, causing a sudden increase in database pressure or even a database crash, which in turn triggers a cascading failure of the entire system, i.e., "cache penetration" or even "cache avalanche".

[0046] To avoid the above situation, the target caching system can dynamically determine the popularity of each cached data based on the access records of the cached data, and then dynamically adjust the lifespan of each cached data.

[0047] Specifically, the target caching system can execute step S101 above, which involves retrieving access records of cached data related to member services from the target caching system's cache within a preset time period. The preset time period includes the current time and / or a preset duration preceding the current time.

[0048] In this embodiment, the target caching system is JetCache, a Java-based caching system wrapper that simplifies caching by providing a unified API (Application Programming Interface) and annotations. It supports local caching (e.g., Caffeine, LinkedHashMap) and remote caching (e.g., Redis), and possesses rich features such as automatic refresh, cache statistics, and distributed locks. The following description uses JetCache as the target caching system to illustrate the solution.

[0049] In actual business operations, the popularity of cached data related to member services in JetCache's cache changes, and the hot data in each cache also changes. Therefore, in order to dynamically manage cached data, access records of each cache can be periodically retrieved, and popularity analysis can be performed based on the access records retrieved in the current period.

[0050] Based on this, JetCache can retrieve access records of cached data within a preset time period, from the current time to / or within a preset time period prior to the current time.

[0051] The preset time period can be set according to actual business needs. It can be 10 seconds or 1 minute, both of which are reasonable and no specific limitation is made here.

[0052] Furthermore, the access records of each cached data at the current time are the real-time access logs of each cached data; the access records of each cached data within a preset time period before the current time are the historical access records of each cached data within that preset time period before the current time. This preset time period can be set according to actual needs, such as 1 day, 3 days, 7 days, etc., all of which are reasonable and are not specifically limited here.

[0053] In one implementation, a CacheInterceptor can be encapsulated in JetCache. This CacheInterceptor can be used to read access requests for various cached data received by the server and record the request information in the access record. JetCache can then obtain access records of cached data related to member services within the current time and / or a preset time period before the current time according to a preset time period.

[0054] In one implementation, JetCache can use a sliding window with a preset data duration to obtain access records of cached data within the current time and the preset time period before the current time. For example, JetCache can use a sliding window with a data duration of 5 minutes to slide on the access records. If the sliding window slides every 10 seconds, JetCache can obtain the access records of cached data within the current time and the 5 minutes before the current time recorded in the access records every 10 seconds, and then calculate the hot data in the cached data within the sliding window.

[0055] JetCache can then perform the above step S102, which is to determine the popularity of each cached data based on the access information of each cached data included in the target access record and the membership business information of the user accessing each cached data, and to determine the hot data in each cached data based on the popularity of each cached data, wherein the target access record is the access record obtained in the current time period.

[0056] The membership service information of the users accessing the cached data may include the membership level of the users accessing the cached data, the value of the membership services involved, and other information.

[0057] In one implementation, JetCache can use cached data whose popularity exceeds a preset threshold as popularity data.

[0058] In another implementation, JetCache can sort the cached data in descending order of popularity and use the top 30% of the cached data as the popularity data.

[0059] In one implementation, after identifying the hot data in each cached data set, JetCache can create and periodically update a list of hot data sets, and dynamically process each cached data set based on this list.

[0060] By dynamically acquiring access records for each cached data and calculating the popularity of each cached data based on the access information of each cached data and the membership business information of the users accessing each cached data, the hot data in each cached data can be determined based on the popularity of each cached data. This can adaptively identify newly emerging high-access data according to changes in actual data access traffic. Compared with the method of manually marking hot data, hot data can be identified in a timely and accurate manner, avoiding the identification of hot data lagging behind actual traffic.

[0061] JetCache can then execute step S103, which involves adjusting the lifespan of hot data to the target lifespan if the lifespan of hot data is not the same as the target lifespan corresponding to the popularity of that hot data. The target lifespan of hot data is positively correlated with the popularity of that hot data.

[0062] To avoid cache breakdown caused by frequent expiration of hot data, which in turn leads to a surge in database access, the target lifespan of hot data is determined based on its popularity. The target lifespan of hot data is positively correlated with its popularity, meaning that data with higher popularity has a longer target lifespan.

[0063] For example, the popularity of cached data is represented by a percentage score. The default fixed TTL for each cached data is 30 minutes. The target lifespan for hot data with a popularity score of less than 95 and not less than 85 is set to 1 hour, and the target lifespan for hot data with a popularity score of not less than 95 is set to 5 hours.

[0064] The cache can also store annotations for each cached data item, which are used to record the lifespan. After identifying hot data, JetCache can read the annotations of the hot data to determine whether the lifespan of the hot data recorded in the annotations is the target lifespan corresponding to the popularity of the hot data. Then, if the lifespan of the hot data is not the target lifespan corresponding to its popularity, JetCache can adjust the lifespan of the hot data to the target lifespan.

[0065] Correspondingly, if the lifespan of hot data is the target lifespan corresponding to the popularity of that hot data, JetCache will not adjust the lifespan of that hot data.

[0066] For example, the popularity of each cached data item can be calculated using the following program fields:

[0067] Public long calculateTTL (double hotScore){

[0068] If (hotScore > 80) return 3600; / / Hotspot data survives for 1 hour;

[0069] else If (hotScore>50) return 1800; / / Medium-heat data survival time is 30 minutes;

[0070] else return 300; / / Normal data survival time is 5 minutes;

[0071] }

[0072] For hot data A with a popularity score of 80, the survival time of hot data A is 30 minutes. Since the survival time of hot data A is not the target survival time of 1 hour corresponding to the popularity score (80 points), the survival time of hot data A is adjusted to 1 hour.

[0073] For cached data B with a popularity score of 60, the lifespan of cached data B is 30 minutes. Since the lifespan of cached data B is the target lifespan of 30 minutes corresponding to the popularity score (60 points) of the hot data, JetCache does not need to adjust the lifespan of cached data B.

[0074] For cached data C with a popularity score of 10, the lifespan of cached data C is 30 minutes. Since the lifespan of cached data C is not the target lifespan of 5 minutes corresponding to the popularity score (10 points) of the hot data, JetCache can adjust the lifespan of the hot data C to 5 minutes.

[0075] Furthermore, JetCache can execute step S104, which means that if the actual lifespan of the hot data has not reached the target lifespan, the hot data will continue to be stored in the cache, and if the actual lifespan of the hot data has reached the target lifespan, the hot data will be deleted from the cache.

[0076] Whether cached data continues to be stored in the cache depends on whether its actual lifespan reaches the lifespan recorded in the annotation. If the actual lifespan of frequently accessed data does not reach the target lifespan, the data continues to be stored in the cache. When the server receives a request for accessing the frequently accessed data, it can retrieve the data from the cache.

[0077] By extending the lifespan of frequently accessed data, the cached data is less likely to expire frequently. When a request for this data is received, the server can directly access it from the cache without retrieving it from the database, reducing database access pressure. Furthermore, because frequently accessed data doesn't expire often, the target caching system also doesn't need to frequently read and cache it from the database, further reducing database access pressure. This achieves the goal of avoiding cache breakdown and database access surges caused by frequent expiration of frequently accessed data, ensuring a high hit rate for frequently accessed data and shortening its response time.

[0078] Correspondingly, if the actual lifespan of hot data reaches the target lifespan, the hot data will be deleted from the cache.

[0079] As one embodiment of this application, the target caching system includes a multi-dimensional hotspot identification engine and a caching system core. The cache also stores annotations for each cached data, and the annotations are used to record the lifespan.

[0080] Step S102 above, namely, determining the popularity of each cached data based on the access information of each cached data included in the target access record and the membership service information of the users accessing each cached data, and determining the hot data in each cached data based on the popularity of each cached data, may include:

[0081] The multi-dimensional hotspot identification engine determines the popularity of each cached data based on the access information of each cached data included in the target access record and the membership business information of the users accessing each cached data. Based on the popularity of each cached data, it identifies the hot data in each cached data and sends the hot data to the cache system core.

[0082] Accordingly, step S103 above, namely, adjusting the survival time of the hot data to the target survival time when the survival time of the hot data is not the same as the target survival time corresponding to the popularity of the hot data, may include:

[0083] The core of the caching system reads the lifespan of the hot data recorded in the annotation; if the lifespan of the hot data is not the target lifespan corresponding to the popularity of the hot data, the lifespan of the hot data recorded in the annotation is modified to the target lifespan.

[0084] The target caching system can include a multi-dimensional hotspot identification engine and a caching system core; that is, JetCache can include a multi-dimensional hotspot identification engine and a JetCache core.

[0085] The multi-dimensional hotspot identification engine can determine the popularity of each cached data after obtaining the target access record, based on the access information of each cached data included in the target access record and the membership business information of the users who access each cached data.

[0086] Then, based on the popularity of each cached data, the hot data in each cached data is calculated, and the identified hot data is sent to the cache system core.

[0087] The caching system core can receive frequently accessed data and read the lifespan of this data recorded in annotations. If the read lifespan of the frequently accessed data is not the target lifespan corresponding to its popularity, the caching system core can modify the lifespan recorded in the annotations to match the target lifespan.

[0088] In this embodiment, the multi-dimensional hotspot identification engine can determine hot data based on the popularity of each cached data in each cache. The core of the caching system can modify the lifespan of hot data in the annotation, thereby achieving the purpose of dynamically adjusting the lifespan of cached data based on data popularity.

[0089] As one embodiment of this application, the method for processing cached data provided in this application embodiment may further include:

[0090] If the amount of each cached data exceeds a preset threshold, cached data of a target size is deleted based on the popularity of each cached data, wherein the target size is the difference between the amount of each cached data and the preset threshold.

[0091] Since the amount of data that can be stored in the cache is fixed, in order to maintain normal access to the cache, it is necessary to control the cache memory usage and automatically delete some data when the cache space is insufficient to avoid memory overflow.

[0092] In the current caching strategy, all cached data share the cache space. When the amount of cached data exceeds a preset threshold, high-value data and ordinary data are deleted indiscriminately, affecting access to high-value data.

[0093] To avoid deleting high-value cached data, if the amount of each cached data exceeds a preset threshold, the difference between the amount of each cached data and the preset threshold is determined to obtain the amount of data to be deleted. Based on the popularity of each cached data, the amount of cached data to be deleted is equal to the target amount of data.

[0094] In one implementation, the cached data is sorted in ascending order of popularity, and the cached data at the top of the list that has the target data size is deleted. This way, cached data with low popularity and the target data size can be deleted, thereby alleviating cache pressure.

[0095] In one implementation, the data value of each cached data can be determined based on its popularity, and the cached data can be arranged in order of increasing data value. The cached data at the top of the list that has the target data size can be deleted. In this way, cached data with low data value and the target data size can be deleted to alleviate cache pressure.

[0096] In this embodiment, by determining the cached data to be evicted based on the popularity of each cached data, and prioritizing the retention of high-value data, it is possible to avoid indiscriminately evicting hot data, thereby preventing cache breakdown caused by the evicting of hot data and subsequent surges in database access. This approach can ensure the cache hit rate of hot data while avoiding cache overflow, thus improving data access efficiency and request response speed.

[0097] As one embodiment of this application, the target caching system includes a caching system core and an adaptive caching strategy engine.

[0098] Accordingly, the step of deleting cached data of a target size based on the popularity of each cached data item may include:

[0099] The adaptive caching strategy engine arranges the cached data in the cache according to a preset order based on the popularity of each cached data; according to the arrangement order, it determines the cached data whose data size is the target data size as the cached data to be deleted, and sends the data identifier of the cached data to be deleted to the cache system core; the cache system core deletes the cached data with the data identifier from the cache.

[0100] After calculating the popularity of each cached data, the adaptive caching strategy engine can arrange the cached data in the cache according to the preset order of popularity. The preset order of popularity can be from high to low or from low to high, and no specific limitation is made here.

[0101] After sorting the cached data, the adaptive caching strategy engine can determine the cached data whose size is the target size according to the sorting order, and use it as the cached data to be deleted.

[0102] In order to continue storing frequently accessed cached data in the cache, and adapting to the aforementioned preset order, the adaptive caching strategy engine can determine the target amount of less frequently accessed cached data as cached data to be deleted, based on the obtained sorting order.

[0103] For example, if the preset order is from high to low, the adaptive caching strategy engine can determine the cached data with the target data volume at the end of the obtained sorting order as the cached data to be deleted; while if the preset order is from low to high, the adaptive caching strategy engine can determine the cached data with the target data volume at the beginning of the obtained sorting order as the cached data to be deleted.

[0104] The adaptive caching strategy engine then sends the data identifier of the cached data to be deleted to the caching system core. The caching system core can receive the data identifier of the cached data to be deleted, identify the cached data with the aforementioned data identifier from the cache, and delete the cached data with the data identifier.

[0105] As one implementation of this application, the cache area includes at least a pre-divided core data pool, which is used to cache core business-related data. Correspondingly, the cache data processing method provided in this application may further include:

[0106] If the current time reaches a preset time node, the core data pool is expanded, wherein the preset time node is the time node before the start of a preset hot event; during the expansion process, the original cached data in the expanded area of ​​the core data pool is migrated to a non-core data pool.

[0107] In the current caching system, core business-related data and ordinary data share the same cache space, and the cache capacity is evenly distributed. The cache configuration cannot be optimized according to different business scenarios, which affects the cache utilization and hit rate, resulting in a long response time for core business data interfaces.

[0108] To avoid the above situation, a core data pool can be pre-allocated in the cache area based on the importance of the business. This core data pool is used to cache data related to core business operations. In one implementation, JetCache can determine the capacity allocation of the core data pool by defining core transaction paths and monitoring the access volume of these paths in real time, and then cache core business-related data in the core data pool.

[0109] In one implementation, the cache area can be pre-divided into a core data pool (corePool), a normal data pool (normalPool), and a temporary data pool (tempPool). The core data pool is set to have a capacity of 50% of the cache capacity, the normal data pool to have a capacity of 30% of the cache capacity, and the temporary data pool to have a capacity of 20% of the cache capacity. For example, the core data pool is used to store core business-related data such as payment data and rights data, while the normal data pool is used to store browsing data and other data.

[0110] During a pre-set hot event, in order to prevent a sudden surge in traffic to core business-related data from filling up the cache space allocated to that core business-related data and causing a decrease in access efficiency, the core data pool can be expanded in advance at a pre-set time point before the pre-set hot event begins.

[0111] Specifically, if the current time is detected to have reached a preset time node, the core data pool can be expanded.

[0112] Among them, preset hot events are pre-set events that will lead to the growth of core businesses, such as anniversary celebrations and holiday promotions. The preset time node before the start of the preset hot event can be set according to actual needs. It can be midnight the day before the start of the preset hot event, or one hour before the start of the preset hot event, etc., without specific limitations.

[0113] In one implementation, the start time of a preset hot event and the preset time node rule for triggering the expansion of the core data pool can be stored as time rules in the JetCache cache. Then, according to the time rules, the expansion of the core data pool can be automatically triggered at a preset time point before the start time of the preset hot event.

[0114] Specifically, JecCache can allocate cache nodes to core data pools, regular data pools, and temporary data pools based on the cached data stored on each node. When expanding the core data pool, the JecCache core can identify the target cache node from the regular and temporary data pools that should be moved to the core cache pool. The data in this target cache node can then be migrated to other cache nodes in the regular and temporary data pools. The target cache node is then cleared to store core business-related cached data that needs to be stored in the core data pool. This expands the cache space used to store core business-related cached data, thus achieving the expansion of the core cache pool.

[0115] Furthermore, a pre-defined mapping between preset hot events and capacity expansion can be established. For example, the capacity expansion for each monthly promotional day could be set to 10% of the cache capacity, and the capacity expansion for an anniversary celebration could be set to 30% of the cache capacity, etc. No specific limitations are specified here. In this way, the expansion capacity for the core data pool can be determined based on the preset hot events and the pre-defined mapping between these events and the expansion capacity, and the core data pool can be expanded accordingly.

[0116] During the expansion of the core data pool, some non-core data pools need to be allocated to the core data pool. The area allocated from the non-core data pool to the core data pool is the expanded area of ​​the core data pool. Therefore, the original cached data stored in the expanded area of ​​the core data pool is non-core data, and the original cached data stored in the expanded area of ​​the core data pool needs to be migrated to the non-core data pool.

[0117] For example, the capacity ratios of the core data pool, the ordinary data pool, and the temporary data pool are 50%, 30%, and 20%, respectively. The core data pool capacity is set to 50% of the cache capacity, the ordinary data pool capacity to 30% of the cache capacity, and the temporary data pool capacity to 20% of the cache capacity. The preset hot event is the anniversary celebration, with the anniversary celebration starting at 00:00 on July 1st. The core data pool is expanded one hour before the anniversary celebration begins, and its cache capacity is increased by 30% before the anniversary celebration begins.

[0118] Thus, when the current time reaches 23:00 on June 30th, the electronic device will expand the core data pool cache capacity by 30%, so that the expanded core data pool capacity is 80% of the cache capacity. The expanded area of ​​the core data pool originally contained the ordinary data pool and the temporary data pool. The original cached data belonging to the ordinary data pool that was originally stored in the expanded area of ​​the core data pool will be migrated to the current ordinary data pool, and the original cached data belonging to the temporary data pool will be migrated to the current temporary data pool.

[0119] In this embodiment, by expanding the core data pool before preset hot events, the cache resources can be dynamically allocated according to the importance of the data and the business scenario. In this way, by adjusting the capacity ratio of each data pool in real time, the access efficiency of core business-related data can be improved, thereby improving the execution efficiency of core business.

[0120] As one implementation method of this application, such as Figure 2 As shown in the embodiments of this application, a method for processing cached data may further include:

[0121] S201: Based on the historical data access records corresponding to the preset hot events, determine the target data with a popularity higher than the first preset popularity threshold from the data corresponding to the preset hot events in the database.

[0122] S202: Load the target data into the expanded core data pool.

[0123] During the period of a pre-set hot event, there may be a surge in access to data in the database that is not written to the cache. Therefore, in order to avoid the impact of the surge in database data access on data access efficiency, this part of the data can be pre-loaded into the cache.

[0124] JetCache can obtain historical data access records corresponding to preset hot events and analyze these records. Based on the access information of the data corresponding to the preset hot events and the membership business information of the users accessing each data, it can determine the popularity of the data corresponding to the preset hot events. Then, based on the popularity of each data, it can identify target data whose popularity exceeds a first preset threshold during the preset hot event and load the target data into the expanded core data pool.

[0125] The first preset popularity threshold can be set according to actual data management needs. When popularity is expressed by a score, the first preset popularity threshold can be 50 points or 70 points, without any specific limitation.

[0126] The target data mentioned above refers to the data corresponding to the preset hot events recorded in the database, which has not been loaded into the cache and is located within the data corresponding to the preset hot events.

[0127] For membership management systems, the peak membership services and access data during peak events remain largely unchanged across different business cycles. Therefore, during the current business cycle, historical access records corresponding to preset peak events can be used to identify target data with high access volume during those events. This target data can then be pre-loaded into the cache to mitigate the impact on the database from surges in access volume during the preset peak events.

[0128] For example, if the most popular membership services remain largely unchanged during each year's anniversary celebration, then the most popular access data during the anniversary celebration will also remain largely unchanged. Before this year's anniversary celebration, the target data with high access volume during this year's anniversary celebration can be predicted based on the historical data access records corresponding to the anniversary celebration. For example, if it is determined that data related to membership prepaid services is the target data with high access volume during this year's anniversary celebration, then the determined target data can be pre-loaded into the cache.

[0129] In one implementation, the LRU (Least Recently Used) model can be used to analyze the historical data access records corresponding to preset hot events, thereby identifying target data in the database that has a popularity higher than a first preset popularity threshold, and loading the target data into the expanded core data pool.

[0130] In one implementation, to avoid a surge of business requests during the start time of a preset hotspot event and the first time period after the start time, and / or during the time period corresponding to a preset business event within the preset hotspot event, which could cause a spike in data access and affect the database's data reading efficiency, the historical data access records corresponding to the preset hotspot event can be analyzed to determine the target data in the database corresponding to the preset hotspot event whose popularity exceeds a first preset popularity threshold during the start time of the preset hotspot event and the first time period after the start time, and / or during the time period corresponding to a preset business event within the preset hotspot event. This target data is then loaded into the expanded core data pool.

[0131] For example, if the preset hot event is the anniversary promotion, JetCache can use the LRU model to predict the target data related to "new users' business fee reduction benefits during the anniversary promotion stored in the database" that has a popularity higher than the first preset popularity threshold in the data corresponding to the anniversary promotion in the database within the first hour after the anniversary promotion starts, based on the historical data access records corresponding to the anniversary promotion in previous years. The target data will be loaded into the core data pool in advance before the anniversary promotion starts.

[0132] For example, a preset hot event is the summer promotion, and the preset business event for the summer promotion is a 50% discount on product prices within the first four hours after the promotion begins. JetCache can use the LRU model to predict the target data in the database corresponding to the summer promotion that has a popularity higher than a first preset popularity threshold for "product price halving" within the first four hours after the summer promotion begins, based on historical data access records corresponding to summer promotions in previous years. The target data is then preloaded into the core data pool before the summer promotion begins.

[0133] In one implementation, after determining the hot data in each cached data based on the popularity of the access records obtained in the current time period, if the hot data is not in the core data pool, the hot data is migrated to the core data pool.

[0134] In addition, deep learning models can be used to optimize hotspot prediction models such as the LRU model, thereby improving the accuracy of hotspot data preloaded into the core data pool.

[0135] In this embodiment, the cache pool is divided according to business priority. During the preset hot events, cache resources are automatically tilted towards the core transaction path, and predicted hot data is preloaded, which can improve data reading efficiency and enhance cache hit rate and system stability in high-concurrency scenarios.

[0136] As one embodiment of this application, the target caching system includes a preset cache interceptor and a multi-dimensional hotspot identification engine. For example... Figure 3 As shown,

[0137] Before step S101 above, i.e., the step of obtaining access records of member business-related cache data in the cache of the target cache system within a preset time period according to a preset time cycle, the cache data processing method provided in this application embodiment may further include:

[0138] S301: The preset cache interceptor reads the access requests for each cached data received by the server and records the request information of the access requests in the access record.

[0139] The request information includes the cached data to be accessed by the access request, the access time, and the membership service information of the user corresponding to the access request.

[0140] The target caching system can include a preset cache interceptor and a multi-dimensional hotspot identification engine, that is, JetCache includes a preset cache interceptor and a multi-dimensional hotspot identification engine.

[0141] The default cache interceptor can read the access requests for each cached data received by the server and obtain the request information for each access request. The request information includes the cached data to be accessed by the access request, the access time, and the membership service information of the user corresponding to the access request.

[0142] After determining the request information for each access request, the request information for each access request can be recorded in the access record so that the cached data can be managed in the future based on the access record.

[0143] Accordingly, step S101 above, namely the step of obtaining access records of cached data related to member services in the cache of the target caching system within a preset time period according to a preset time cycle, may include:

[0144] S302: The multi-dimensional hotspot identification engine obtains access records of cached data related to member services in the cache of the target cache system within a preset time period according to a preset time cycle.

[0145] The multi-dimensional hotspot identification engine can obtain access records of cached data related to member business in the cache of the target cache system within a preset time period.

[0146] Accordingly, step S102 above, namely, determining the popularity of each cached data based on the access information of each cached data included in the target access record and the membership service information of the users accessing each cached data, and determining the hot data in each cached data based on the popularity of each cached data, may include:

[0147] S303: The multi-dimensional hotspot identification engine determines the access popularity of each cached data based on the cached data to be accessed by each access request included in the target access record and the access time, and determines the user value corresponding to each cached data based on the user's membership business information included in the target access record.

[0148] S304: Calculate the popularity of each cached data based on the access popularity and user value represented by the access records obtained in the current time period.

[0149] S305: The cached data with a popularity higher than the second preset popularity threshold is designated as hot data.

[0150] S306: If the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, adjust the survival time of the hot data to the target survival time.

[0151] The target lifespan of the hot data is positively correlated with its popularity.

[0152] S307: If the actual lifespan of the hot data does not reach the target lifespan, continue to store the hot data in the cache; if the actual lifespan of the hot data reaches the target lifespan, delete the hot data from the cache.

[0153] For each preset time period, the multi-dimensional hotspot identification engine can determine the access popularity of each cached data based on the cached data to be accessed and the access time of each access request included in the target access record obtained in the current time period.

[0154] Among them, access popularity can be represented by the number of accesses and the last access time. Therefore, the multidimensional hotspot identification engine can determine the number of accesses of each cached data in the target access record based on the cached data to be accessed by each access request and the access time included in the target access record, and determine the last access time of each cached data, thereby determining the access popularity corresponding to each cached data.

[0155] The multi-dimensional hotspot identification engine can also determine the user value corresponding to each cached data based on the user's membership business information included in the target access record.

[0156] User value can be represented by user level and transaction amount. Therefore, the multidimensional hotspot recognition engine can determine the user level of the user accessing each cached data and the transaction amount involved in this access based on the user's membership business information included in the target access record, thereby determining the user value corresponding to each cached data.

[0157] Furthermore, the multi-dimensional hotspot identification engine can calculate the popularity of each cached data based on its access popularity and user value, and identify cached data whose popularity exceeds the second preset popularity threshold as hotspot data.

[0158] The second preset popularity threshold can be set according to actual data management needs. When popularity is expressed by a score, the second preset popularity threshold can be 60 points, 75 points, etc., without specific limitations.

[0159] Step S306 is the same as step S103, and step S307 is the same as step S104. The specific implementation methods of steps S306-S307 are the same as the specific implementation methods of steps S103-S104 mentioned above, and will not be repeated here.

[0160] In this embodiment, through the above settings, the electronic device can quickly and accurately identify hot data with high access popularity and high user value by determining the hot data based on the access popularity and user value corresponding to each cached data, thereby improving the efficiency of hot data identification.

[0161] As one implementation of this application, access popularity includes the number of visits and the time of the last visit; user value includes user level and transaction amount.

[0162] like Figure 4 As shown, step S304 above, namely the step of calculating the popularity of each cached data based on the access records obtained in the current time period and the corresponding access popularity and user value, may include:

[0163] S401: Calculate the access frequency score for each cached data based on the number of accesses corresponding to each cached data.

[0164] S402: Calculate the value score of each cached data based on the user level and transaction amount corresponding to each cached data;

[0165] S403: Calculate the timeliness score of each cached data based on the last access time of each cached data;

[0166] S404: For each cached data, calculate the popularity of the cached data based on its access frequency score, value score, and timeliness score.

[0167] For each cached data, the access popularity of the cached data can include the number of accesses and the time of the last access; the user value of the cached data can include the user level and the transaction amount.

[0168] JetCache can extract the access count, last access time, user ID, and transaction amount corresponding to the data identifier of each cached data from the access records obtained in the current time period, and determine the user level based on the user ID.

[0169] Then, JetCache can calculate the access frequency score of each cached data based on the number of accesses corresponding to each cached data; calculate the value score of each cached data based on the user level and transaction amount corresponding to each cached data; and calculate the timeliness score of each cached data based on the last access time of each cached data.

[0170] Furthermore, for each cached data, JetCache can calculate the popularity of that cached data based on its access frequency score, value score, and timeliness score.

[0171] In one implementation, JetCache can directly calculate the sum of the access frequency score, value score, and timeliness score of each cached data, which serves as the popularity of that cached data.

[0172] In another implementation, JetCache can calculate the weighted sum of the access frequency score, value score, and timeliness score of each cached data point as the "hotness" of that cached data. Furthermore, the weights of the access frequency score, value score, and timeliness score can be set according to actual needs, without specific limitations here. For example, the weights of the access frequency score, value score, and timeliness score could be 30%, 30%, and 40%, respectively.

[0173] For example, the popularity of each cached data item can be calculated using the following program fields:

[0174] / / Calculate the score for cached data (0-100 points);

[0175] Public double calculateHotScore (String dateKey, AccessLog log){

[0176] / / Dimension 1: Access frequency score (30% weight);

[0177] double freqScore =calculateFrequencyScore(dateKey); / /

[0178] / / Dimension 2: Value Score (40% weight);

[0179] double valueScore=calculateValueScore(log.getVipLevel(),log.getAmount()); / /

[0180] / / Dimension 3: Access Timeliness (30% weight);

[0181] double timeScore = calculateTimeScore(log.getAccessTime()); / / Recent accesses have higher weight.

[0182] Return freqScore×0.3+valueScore×0.4+timeScore×0.3;

[0183] }

[0184] JetCache's CacheInterceptor can intercept access records during data access and calculate scores in real time based on a sliding window. Based on the access records within 1 hour before the current time, it is determined that the cached data "Diamond Member (User Level 5) purchased an annual membership (amount 298 yuan)" was accessed 10 times, with the last access time being 10 seconds ago, and the total number of data accesses being 20. The value score (valueScore=90), access frequency score (freqScore=80), and access timeliness score (timeScore=90) of the cached data are calculated, and the final hotScore=95, marking the cached data as hot data.

[0185] In this embodiment, the popularity of each cached data is quickly and accurately determined based on the number of accesses, the last access time, the user level, and the transaction amount. By integrating the three-dimensional features of access frequency, data value, and access timeliness, the one-sidedness of relying solely on frequency to determine data popularity is solved, thus improving the accuracy of hotspot identification.

[0186] As mentioned earlier, after calculating the popularity of each cached data, the adaptive caching strategy engine can arrange the cached data in the cache according to the preset order of popularity of each cached data; wherein, the preset order of popularity of each cached data can be in order of popularity from high to low, or in order of popularity from low to high, and no specific limitation is made here.

[0187] After sorting the cached data, the adaptive caching strategy engine can determine the cached data whose size is the target size according to the sorting order, and use it as the cached data to be deleted.

[0188] In order to continue storing frequently accessed cached data in the cache, and adapting to the aforementioned preset order, the adaptive caching strategy engine can determine the target amount of less frequently accessed cached data as cached data to be deleted, based on the obtained sorting order.

[0189] For example, if the preset order is from high to low, the adaptive caching strategy engine can determine the cached data with the target data volume at the end of the obtained sorting order as the cached data to be deleted; while if the preset order is from low to high, the adaptive caching strategy engine can determine the cached data with the target data volume at the beginning of the obtained sorting order as the cached data to be deleted.

[0190] The adaptive caching strategy engine then sends the data identifier of the cached data to be deleted to the caching system core. The caching system core can receive the data identifier of the cached data to be deleted, identify the cached data with the aforementioned data identifier from the cache, and delete the cached data with the data identifier.

[0191] In one implementation, the electronic device determines the access frequency score, value score, and timeliness score of each cached data, and calculates the popularity of each cached data based on the access frequency score, value score, and timeliness score. Based on the popularity and user value of each cached data, it calculates the data value of each cached data, and deletes cached data with low data value whose data volume is equal to the target data volume, according to the data value of each cached data.

[0192] For example, weights can be assigned to popularity and user value, with popularity having a weight of 0.6 and user value having a weight of 0.4. For each cached data point, the popularity and user value are weighted and summed to obtain the data value of that cached data. The data values ​​of every two cached data points are compared, and the cached data points are sorted in ascending order. Since the cached data is sorted from high to low based on its data value, real-time load monitoring allows for the prioritization of discarding ordinary browsing data with a data value less than 30 points when the ratio of used cache capacity to total cache capacity indicates a high cache load. This prioritizes the retention of high-value cached data such as payment records.

[0193] In this embodiment, when the amount of each cached data exceeds a preset threshold, the electronic device evicts data based on the popularity of each cached data. While ensuring the normal operation of the cache, it improves the hit rate and survival rate of high-popularity data, increases the data interface success rate of high-popularity data, avoids increased database access pressure caused by the eviction of popular data, reduces database load, improves data access efficiency and business execution efficiency, improves user experience, and reduces user complaints.

[0194] As one embodiment of this application, step S401 above, namely the step of calculating the access frequency score of each cached data based on the access count corresponding to each cached data, may include:

[0195] For each cached data, calculate the ratio of the number of accesses to the total number of accesses to obtain the access percentage of the cached data. Based on the access percentage of the cached data and the preset correspondence between the access percentage and the access frequency score, calculate the access frequency score of the cached data.

[0196] JetCache can determine the number of accesses for each cached data and the total number of data accesses based on the access records obtained in the current time period. Then, for each cached data, it calculates the ratio of the number of accesses for that cached data to the total number of data accesses, thus obtaining the access percentage of that cached data.

[0197] Furthermore, for each cached data, the access frequency score is calculated based on the access frequency ratio of that cached data and the preset correspondence between the access frequency ratio and the access frequency score, thus obtaining the access frequency score for each cached data.

[0198] Among them, the access frequency score of each cached data is positively correlated with the proportion of access times of that cached data, that is, the higher the access time of the cached data with the higher proportion of access times, the higher the access frequency score.

[0199] For example, cached data with an access frequency ratio of 0.03 has an access frequency score of 3 points; cached data with an access frequency ratio of 0.3 has an access frequency score of 30 points.

[0200] In this embodiment, the access frequency score of each cached data can be calculated quickly and accurately through the above settings. Then, the popularity of each cached data can be calculated quickly, and each cached data can be managed according to its popularity.

[0201] As one embodiment of this application, step S402 above, namely the step of calculating the value score of each cached data based on the user level and transaction amount corresponding to each cached data, may include:

[0202] For each cached data, the user level score of the cached data is calculated based on the user level corresponding to the cached data and the preset correspondence between user level and user level score.

[0203] Based on the transaction amount corresponding to the cached data, and the preset correspondence between the transaction amount and the transaction amount score, the transaction amount score of the cached data is calculated.

[0204] The value score of the cached data is calculated based on the user level score and transaction amount score.

[0205] JetCache can determine the user level and transaction amount corresponding to each cached data based on the access records obtained in the current time period.

[0206] Furthermore, for each cached data, JetCache can calculate the user level score based on the user level corresponding to that cached data and the preset correspondence between user levels and user level scores. The user level and user level score are positively correlated; for example, user levels are represented by stars, with higher stars indicating higher user levels. A one-star user has a user level score of 20 points, and a five-star user has a user level score of 100 points.

[0207] Furthermore, JetCache can calculate the transaction amount score of the cached data based on the transaction amount corresponding to the cached data and the preset correspondence between transaction amount and transaction amount score. For ease of calculation, the transaction amount score can be determined based on the amount range in which the transaction amount falls; thus, the correspondence between transaction amount and transaction amount score is the transaction amount score corresponding to each transaction amount range.

[0208] For example, if the transaction amount is in the first range, then the transaction amount score is 10 points; if the transaction amount is in the fifth range, then the transaction amount score is 50 points; if the transaction amount is in the eighth range, then the transaction amount score is 80 points, and so on.

[0209] Furthermore, for each cached data, a value score is calculated based on the user level score and transaction amount score of that cached data.

[0210] To calculate the value score, the user level score and transaction amount score should be based on the same scoring system. If the user level score and transaction amount score are not based on the same scoring system, they should be converted to the same scoring system before calculating the value score. For example, if the user level score is 80 points on a 100-point scale and the transaction amount score is 5 points on a 10-point scale, the transaction amount score should be converted to 50 points on a 100-point scale before calculating the value score.

[0211] In one implementation, for each cached data, the weighted sum of the user level score and transaction amount score of that cached data is calculated as the value score of that cached data.

[0212] In one implementation, for each cached data point, the sum of its user level score and transaction amount score is calculated. Then, based on the maximum score corresponding to each of the user level and transaction amount scores, this sum is converted into a value score corresponding to those scores, which is used as the value score of the cached data. For example, if the cached data has a user level score of 80 points and a transaction amount score of 100 points, the sum of these two scores is 180 points, and the total score is 200 points. Therefore, 180 points can be converted to 90 points on a percentage scale, meaning the value score of the cached data is 90 points.

[0213] In this embodiment, the value score of each cached data can be calculated quickly and accurately through the above settings, and then the popularity of each cached data can be calculated quickly, and each cached data can be managed according to its popularity.

[0214] As one embodiment of this application, step S403 above, namely the step of calculating the timeliness score of each cached data based on the last access time of each cached data, may include:

[0215] For each cached data, the timeliness score is calculated based on the last access time of the cached data and the preset correspondence between the last access time and the timeliness score.

[0216] For each cached data, the electronic device calculates the timeliness score of the cached data based on the last access time of the cached data and the preset correspondence between the last access time and the timeliness score.

[0217] To facilitate calculation, the timeliness score can be determined based on the time interval of the last access time. The correspondence between the last access time and the timeliness score is the timeliness score for each time interval. For example, if the last access time of cached data falls within the last five seconds, the timeliness score for that cached data is 100 points; if the last access time of cached data falls within a time interval indicated one day ago, the timeliness score for that cached data is 10 points, and so on.

[0218] In this embodiment, the timeliness score of each cached data can be calculated quickly and accurately through the above settings. Then, the popularity of each cached data can be calculated quickly, and each cached data can be managed according to its popularity.

[0219] To facilitate understanding of the cache data processing method provided in this application embodiment, the following description is based on a cache data management example. A schematic diagram of the electronic device used in this cache data management example is shown below. Figure 5 As shown, this electronic device achieves a closed loop of "identification-decision-execution" through an adaptive caching system based on JetCache, thereby managing cached data in the Redis cache cluster.

[0220] The JetCache framework includes the JetCache extended cache interceptor, a multi-dimensional hotspot identification engine, an adaptive caching strategy engine, a resource dynamic allocation engine, and the JetCache core.

[0221] The aforementioned cache interceptor retrieves access records of cached data within a preset time period from the member transaction system, including real-time access logs and historical access data. This process collects access logs (containing user IDs, data keys, member levels, etc.) and triggers a hotspot identification process.

[0222] The multi-dimensional hotspot identification engine determines the hotspot data in each cached data based on the access records obtained in the current time period. It calculates the hotspot data based on access frequency, value, and timeliness, identifies and marks hotspot data (score > 70), and outputs the hotspot data score to the adaptive caching strategy engine.

[0223] The adaptive caching strategy engine dynamically adjusts the TTL (Time to Live) based on the popularity of each cached data, executes a hybrid "value-popularity" eviction policy, and outputs the caching policy (TTL / eviction rule) to the resource dynamic allocation engine and the JetCache core. The JetCache core receives the caching policy, stores the caching policy, and executes the caching policy.

[0224] The dynamic resource allocation engine receives the caching strategy, generates a resource allocation scheme, manages the tiered cache pools (core / normal / temp) in the Redis cache cluster, dynamically adjusts the capacity ratio, preloads and predicts hot spots, implements data storage through the Redis cache cluster, realizes dynamic resource allocation, and outputs the resource allocation scheme to the JetCache core so that the JetCache core can store the resource allocation scheme.

[0225] The multi-dimensional hotspot identification process diagram is as follows: Figure 6 As shown, it specifically includes:

[0226] S601 collects real-time access logs;

[0227] S602, Sliding window aggregation (5-minute window);

[0228] S603, calculate the three-dimensional score (frequency / value / timeliness);

[0229] S604, Hotspot Judgment (data with a popularity score > 70 is marked as hotspot data);

[0230] S605 outputs a list of hotspot data.

[0231] By extending JetCache's CacheInterceptor, access logs are intercepted during data access, and the popularity of each cached data item is calculated in real time based on a sliding window. This sliding window aggregates access data from the past 5 minutes.

[0232] Calculate the scores for each cached data in three dimensions: access frequency score, value score, and timeliness score. Then, sum the scores of these three dimensions in a weighted manner to obtain the popularity score for each cached data.

[0233] Based on the popularity of each cached data, cached data with a popularity score greater than 70 is identified as hot data, and a list of hot data is constructed and output.

[0234] The adaptive caching strategy process is illustrated in the diagram below. Figure 7 As shown, it specifically includes:

[0235] S701, receives a list of hotspot data;

[0236] S702 assigns dynamic survival time to hot data (the higher the popularity, the longer the survival time);

[0237] S703 triggers hybrid eviction when cache capacity > 80%;

[0238] S704, sort by "popularity + value", and eliminate low-priority data;

[0239] S705, JetCache core update cache configuration.

[0240] Receive a list of hot data. If the lifespan of hot data is not the target TTL corresponding to the popularity of the hot data, dynamically adjust the lifespan of the hot data to the target TTL. The higher the popularity score of the hot data, the longer the target TTL of the hot data.

[0241] When the amount of cached data exceeds 80% of the total cache capacity, hybrid eviction is triggered. The JetCache core sorts the cached data according to its "popularity + value," and removes low-priority cached data, thus achieving the goal of eviction based on popularity and value.

[0242] The schematic diagram of the dynamic allocation process for cache resources is as follows: Figure 8 As shown, it specifically includes:

[0243] S801, a primary hierarchical cache pool;

[0244] S802, real-time monitoring of each pool's load and core path access volume;

[0245] S803 automatically expands its corePool to 80% capacity during major sales events;

[0246] S804, invoke the prediction model and preload the target data into the corePool (core data pool).

[0247] S805, non-core data overflows to normalPool (normal data pool) / tempPool (temporary data pool).

[0248] Based on the business scenario, a primary tiered cache pool is established, dividing the cache into corePool (50%), normalPool (30%), and tempPool (20%). Among them, corePool is used to store core business-related data.

[0249] Real-time monitoring of pool load and core path access volume; automatically scaling up the corePool to 80% one hour before the start of a major promotion. Utilizing a predictive model, target data with a popularity exceeding a first preset popularity threshold from data corresponding to preset hot events is preloaded into the corePool, while non-core data is overflowed into the bormalPool / twmpPool.

[0250] In this way, by integrating three-dimensional features of access frequency, data value, and timeliness, the one-sidedness of relying solely on frequency in traditional methods is addressed, thereby improving the accuracy of hotspot identification. The cache lifecycle is dynamically adjusted based on hotspot scores, and eviction is carried out in conjunction with data value, thereby improving the core data hit rate. Furthermore, through scenario-based resource allocation and predictive preloading, the response time of core interfaces during major promotional periods is shortened, thereby improving business response efficiency.

[0251] Based on the same inventive concept, and corresponding to the embodiments provided in this application above... Figure 1 The present application provides a method for processing cached data, and also provides a device for processing cached data.

[0252] Figure 9 This is a schematic diagram of a data caching processing device provided in an embodiment of this application. Figure 9 As shown, the device may include the following modules:

[0253] The access record acquisition module 910 is used to acquire access records of cached data related to member business in the cache of the target cache system within a preset time period according to a preset time cycle, wherein the preset time period includes the current time and / or a preset time period before the current time.

[0254] The hotspot data determination module 920 is used to determine the popularity of each cached data based on the access information of each cached data included in the target access record and the membership business information of the user accessing each cached data, and to determine the hotspot data in each cached data based on the popularity of each cached data, wherein the target access record is the access record obtained in the current time period.

[0255] The survival time adjustment module 930 is used to adjust the survival time of the hot data to the target survival time when the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, wherein the target survival time of the hot data is positively correlated with the popularity of the hot data;

[0256] The processing module 940 is configured to continue storing the hot data in the cache when the actual lifespan of the hot data does not reach the target lifespan, and to delete the hot data from the cache when the actual lifespan of the hot data reaches the target lifespan.

[0257] In the technical solution provided by this invention, the target caching system can dynamically determine the hot data in the cached data related to member services in the target caching system according to a preset time period, and adjust the lifespan of each hot data to a target lifespan corresponding to its popularity. This achieves the purpose of dynamically adjusting the lifespan of cached data based on data popularity. By extending the lifespan of cached data according to its popularity, highly popular data in the cache will not frequently expire. Thus, when an access request for hot data is received, the server can directly access the hot data from the cache without having to read it from the database, reducing the data access pressure on the database. Furthermore, since hot data does not frequently expire, the target caching system does not need to frequently read and cache the hot data from the database, further reducing the data access pressure on the database. This achieves the goal of avoiding cache breakdown caused by frequent expiration of hot data and the problem of a surge in database access pressure, improving the hot data hit rate, and ensuring data access efficiency.

[0258] Optionally, in one specific implementation, the target caching system includes a multi-dimensional hotspot identification engine and a caching system core. The cache also stores annotations for each cached data, and the annotations are used to record the lifespan. The hotspot data determination module 920 includes:

[0259] The first determining submodule is used by the multi-dimensional hotspot identification engine to determine the popularity of each cached data based on the access information of each cached data included in the target access record and the membership business information of the user accessing each cached data, and based on the popularity of each cached data, to determine the hot data in each cached data, and send the hot data to the cache system core.

[0260] The survival time adjustment module 930 includes:

[0261] The duration reading submodule is used by the core of the caching system to read the lifespan of the hot data recorded in the annotation;

[0262] The duration modification submodule is used to modify the duration of the hot data recorded in the annotation to the target duration when the duration of the hot data is not the same as the target duration corresponding to the popularity of the hot data.

[0263] Optionally, in one specific implementation, the apparatus further includes:

[0264] The deletion module is used to delete cached data of a target size based on the popularity of each cached data when the data volume of each cached data exceeds a preset threshold. The target data volume is the difference between the data volume of each cached data and the preset threshold.

[0265] Optionally, in one specific implementation, the target caching system includes a caching system core and an adaptive caching strategy engine; the deletion module includes:

[0266] The sorting submodule is used by the adaptive caching strategy engine to sort the cached data in the cache according to a preset order of the popularity of each cached data; according to the sorting order, the cached data with a data size of the target data size is determined as the cached data to be deleted, and the data identifier of the cached data to be deleted is sent to the cache system core.

[0267] The deletion submodule is used by the core of the caching system to delete cached data with the data identifier from the cache.

[0268] Optionally, in one specific implementation, the cache area includes at least a pre-divided core data pool, which is used to cache core business-related data; the device further includes:

[0269] The expansion module is used to expand the core data pool when the current time is detected to have reached a preset time node, wherein the preset time node is the time node before the start of a preset hot event;

[0270] The migration module is used to migrate the original cached data in the expanded area of ​​the core data pool to a non-core data pool during the expansion process.

[0271] Optionally, in one specific implementation, the apparatus further includes:

[0272] The target data determination module is used to determine target data with a popularity higher than a first preset popularity threshold from the data corresponding to the preset hot events in the database, based on the historical data access records corresponding to the preset hot events.

[0273] The data loading module is used to load the target data into the expanded core data pool.

[0274] Optionally, in one specific implementation, the target caching system includes a preset cache interceptor and a multi-dimensional hotspot identification engine; the device further includes:

[0275] The request reading module is used to read the access requests for each cached data received by the server before obtaining the access records of the cached data related to the member business in the cache of the target cache system within a preset time period according to a preset time cycle, and record the request information of the access request in the access record. The request information includes the cached data to be accessed by the access request, the access time, and the member business information of the user corresponding to the access request.

[0276] The access record acquisition module 910 includes:

[0277] The record acquisition submodule is used by the multi-dimensional hotspot identification engine to acquire access records of cached data related to member services in the cache of the target cache system within a preset time period according to a preset time cycle.

[0278] The hotspot data determination module 920 includes:

[0279] The second determining submodule is used by the multi-dimensional hotspot identification engine to determine the access popularity of each cached data based on the cached data to be accessed by each access request included in the target access record and the access time, and to determine the user value corresponding to each cached data based on the user's membership business information included in the target access record.

[0280] The popularity calculation submodule is used to calculate the popularity of each cached data based on the access popularity and user value corresponding to each cached data.

[0281] The third determining submodule is used to identify cached data whose popularity is higher than the second preset popularity threshold as hot data.

[0282] Optionally, in one specific implementation, the access popularity includes the number of accesses and the time of the last access; the user value includes the user level and the transaction amount; the popularity calculation submodule includes:

[0283] The first calculation unit is used to calculate the access frequency score of each cached data based on the number of accesses corresponding to each cached data.

[0284] The second calculation unit is used to calculate the value score of each cached data based on the user level and transaction amount corresponding to each cached data.

[0285] The third calculation unit is used to calculate the timeliness score of each cached data based on the last access time of each cached data.

[0286] The fourth calculation unit is used to calculate the popularity of each cached data based on its access frequency score, value score, and timeliness score.

[0287] Optionally, in one specific implementation, the first computing unit is specifically used for:

[0288] For each cached data, calculate the ratio of the number of accesses to the total number of accesses to obtain the access percentage of the cached data. Based on the access percentage of the cached data and the preset correspondence between the access percentage and the access frequency score, calculate the access frequency score of the cached data.

[0289] Optionally, in one specific implementation, the second computing unit is specifically used for:

[0290] For each cached data, the user level score of the cached data is calculated based on the user level corresponding to the cached data and the preset correspondence between user level and user level score.

[0291] Based on the transaction amount corresponding to the cached data, and the preset correspondence between the transaction amount and the transaction amount score, the transaction amount score of the cached data is calculated.

[0292] The value score of the cached data is calculated based on the user level score and transaction amount score.

[0293] Optionally, in one specific implementation, the third computing unit is specifically used for:

[0294] For each cached data, the timeliness score is calculated based on the last access time of the cached data and the preset correspondence between the last access time and the timeliness score.

[0295] This application also provides an electronic device, such as... Figure 10 As shown, it includes a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0296] Memory 1003 is used to store computer programs;

[0297] The processor 1001, when executing the program stored in the memory 1003, implements any of the above method steps.

[0298] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0299] The communication interface is used for communication between the aforementioned terminal and other devices.

[0300] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0301] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0302] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the cache data processing method described in any of the above embodiments.

[0303] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the cached data processing methods described in the above embodiments.

[0304] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0305] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0306] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0307] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for processing cached data, characterized in that, Applied to the target caching system; the method includes: According to a preset time period, access records of cached data related to member services in the cache of the target caching system are obtained within a preset time period, wherein the preset time period includes the current time and / or a preset time period before the current time. Based on the access information of each cached data included in the target access record and the membership business information of the users accessing each cached data, the popularity of each cached data is determined, and based on the popularity of each cached data, the hot data in each cached data is determined, wherein the target access record is the access record obtained in the current time period. If the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, the survival time of the hot data is adjusted to the target survival time, wherein the target survival time of the hot data is positively correlated with the popularity of the hot data; If the actual lifespan of the hot data does not reach the target lifespan, the hot data continues to be stored in the cache; if the actual lifespan of the hot data reaches the target lifespan, the hot data is deleted from the cache.

2. The method according to claim 1, characterized in that, The target caching system includes a multi-dimensional hotspot identification engine and a caching system core. The cache also stores annotations for each cached data, and the annotations are used to record the lifespan. The steps of determining the popularity of each cached data based on the access information of each cached data included in the target access record and the membership service information of the users accessing each cached data, and determining the hot data in each cached data based on the popularity of each cached data, include: The multi-dimensional hotspot identification engine determines the popularity of each cached data based on the access information of each cached data included in the target access record and the membership business information of the users accessing each cached data. Based on the popularity of each cached data, it identifies the hot data in each cached data and sends the hot data to the cache system core. The step of adjusting the survival time of the hot data to the target survival time when the survival time of the hot data is not the same as the target survival time corresponding to the popularity of the hot data includes: The core of the caching system reads the lifespan of the hot data recorded in the annotation; If the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, the survival time of the hot data recorded in the annotation shall be modified to the target survival time.

3. The method according to claim 1, characterized in that, The method further includes: If the amount of each cached data exceeds a preset threshold, cached data of a target size is deleted based on the popularity of each cached data, wherein the target size is the difference between the amount of each cached data and the preset threshold.

4. The method according to claim 3, characterized in that, The target caching system includes a caching system core and an adaptive caching strategy engine; The step of deleting cached data of a target size based on the popularity of each cached data includes: The adaptive caching strategy engine arranges the cached data in the cache according to a preset order of the popularity of each cached data; according to the arrangement order, it determines the cached data whose data size is the target data size as the cached data to be deleted, and sends the data identifier of the cached data to be deleted to the cache system core; The core of the caching system deletes cached data with the data identifier from the cache.

5. The method according to claim 1, characterized in that, The cache area includes at least a pre-defined core data pool, which is used to cache core business-related data; the method further includes: If the current time reaches a preset time node, the core data pool is expanded, wherein the preset time node is the time node before the start of a preset hot event; During the expansion process, the original cached data within the expanded area of ​​the core data pool is migrated to the non-core data pool.

6. The method according to claim 5, characterized in that, The method further includes: Based on the historical data access records corresponding to the preset hot events, target data with a popularity higher than the first preset popularity threshold is determined from the data corresponding to the preset hot events in the database. The target data is loaded into the expanded core data pool.

7. The method according to any one of claims 1-6, characterized in that, The target caching system includes a preset cache interceptor and a multi-dimensional hotspot identification engine; Before the step of obtaining access records of member-related cached data in the cache of the target caching system within a preset time period according to a preset time cycle, the method further includes: The preset cache interceptor reads the access requests for each cached data received by the server and records the request information of the access request in the access record. The request information includes the cached data to be accessed by the access request, the access time, and the membership service information of the user corresponding to the access request. The step of obtaining access records of cached data related to member services in the cache of the target caching system within a preset time period according to a preset time cycle includes: The multi-dimensional hotspot identification engine obtains access records of cached data related to member services in the cache of the target caching system within a preset time period according to a preset time cycle. The steps of determining the popularity of each cached data based on the access information of each cached data included in the target access record and the membership service information of the users accessing each cached data, and determining the hot data in each cached data based on the popularity of each cached data, include: The multi-dimensional hotspot identification engine determines the access popularity of each cached data based on the cached data to be accessed by each access request included in the target access record and the access time, and determines the user value corresponding to each cached data based on the user's membership business information included in the target access record. The popularity of each cached data is calculated based on its access frequency and user value. The cached data whose popularity exceeds the second preset popularity threshold is designated as hot data.

8. The method according to claim 7, characterized in that, The access popularity includes the number of visits and the time of the last visit; the user value includes the user level and the transaction amount. The step of calculating the popularity of each cached data based on the access popularity and user value corresponding to each cached data includes: Calculate the access frequency score for each cached data based on the number of accesses corresponding to each cached data. Calculate the value score of each cached data based on the user level and transaction amount corresponding to each cached data. Calculate the timeliness score of each cached data based on the last access time of each cached data. For each cached data, the popularity of the cached data is calculated based on its access frequency score, value score, and timeliness score.

9. The method according to claim 8, characterized in that, The step of calculating the access frequency score of each cached data based on the access count corresponding to each cached data includes: For each cached data, calculate the ratio of the number of accesses to the total number of accesses to obtain the access percentage of that cached data; The access frequency score of the cached data is calculated based on the percentage of accesses to the cached data and the pre-defined correspondence between the percentage of accesses and the access frequency score.

10. The method according to claim 8, characterized in that, The step of calculating the value score of each cached data point based on the user level and transaction amount corresponding to each cached data point includes: For each cached data, the user level score of the cached data is calculated based on the user level corresponding to the cached data and the preset correspondence between user level and user level score. Based on the transaction amount corresponding to the cached data, and the preset correspondence between the transaction amount and the transaction amount score, the transaction amount score of the cached data is calculated. The value score of the cached data is calculated based on the user level score and transaction amount score.

11. The method according to claim 8, characterized in that, The step of calculating the timeliness score of each cached data based on the last access time of each cached data includes: For each cached data, the timeliness score is calculated based on the last access time of the cached data and the preset correspondence between the last access time and the timeliness score.

12. A data processing apparatus for caching data, characterized in that, Applied to a target caching system; the apparatus includes: The access record acquisition module is used to acquire access records of cached data related to member services in the cache of the target caching system within a preset time period according to a preset time cycle, wherein the preset time period includes the current time and / or a preset time period before the current time. The hotspot data determination module is used to determine the popularity of each cached data based on the access information of each cached data included in the target access record and the membership business information of the user accessing each cached data, and to determine the hotspot data in each cached data based on the popularity of each cached data, wherein the target access record is the access record obtained in the current time period. The survival time adjustment module is used to adjust the survival time of the hot data to the target survival time when the survival time of the hot data is not the target survival time corresponding to the popularity of the hot data, wherein the target survival time of the hot data is positively correlated with the popularity of the hot data; The processing module is configured to continue storing the hot data in the cache when the actual lifespan of the hot data does not reach the target lifespan, and to delete the hot data from the cache when the actual lifespan of the hot data reaches the target lifespan.

13. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-11.

Images