Cache object scheduling method and system based on host isolated cache architecture

Abstract

This disclosure provides a method and system for scheduling cached objects based on a host-isolated caching architecture. One embodiment of the method includes: maintaining an object pool and a buffer pool; for each cached object, determining a credit score based on its reuse count and idle time, and determining a reuse count limit for that cached object based on the credit score. The credit score is positively correlated with the reuse count and negatively correlated with the idle time. The reuse count limit is the maximum number of times the cached object is allowed to be reused in the buffer pool; in response to not finding a cached object in the object pool, selecting the cached object with the largest difference between its reuse count limit and reuse count from the buffer pool as the target cached object, migrating the target cached object to the object pool, and returning the target cached object. This disclosure improves the object reuse hit rate while ensuring low-latency acquisition in the active area, reduces garbage collection frequency and memory fragmentation, and simultaneously achieves timely removal of low-value objects, avoiding invalid memory occupation.

Description

Technical Field

[0001] This invention belongs to the field of computer storage technology, and in particular relates to a method and system for scheduling cache objects based on a host-isolated cache architecture. Background Technology

[0002] In systems employing host-isolated caching architectures, cached objects are typically managed in two tiers: an active zone and a backing zone. When migrating objects from the backing zone to the active zone, existing methods generally select objects based on a single total reuse count or a first-in, first-out (FIFO) strategy. The method based on total reuse counts fails to distinguish between historically high-frequency but recently idle objects and historically low-frequency but recently active objects. This results in recently active, high-value objects being prematurely reclaimed, while long-term idle, low-value objects remain in the backing zone. The FIFO strategy completely ignores the value differences between objects, often migrating out objects that have just been evicted but still have reuse potential. This leads to frequent object creation and destruction, increasing garbage collection overhead and memory fragmentation. Furthermore, a fixed eviction threshold cannot adapt to dynamic changes in business load, causing memory waste under low load and ineffective garbage collection under high load. Therefore, accurately selecting the most reusable objects from the backing zone becomes crucial in balancing the contradiction between object reuse hit rate and memory consumption. Summary of the Invention

[0003] This disclosure provides a method and system for scheduling cached objects based on a host-isolated caching architecture. While ensuring low-latency acquisition of the active area, it improves the object reuse hit rate, significantly reduces the creation of new objects, thereby reducing the frequency of garbage collection and memory fragmentation. At the same time, it enables the timely removal of low-value objects and avoids invalid memory occupation.

[0004] This disclosure provides a method for scheduling cache objects based on a host-isolated caching architecture, including: Maintain an object pool and a buffer pool. The object pool is used to store active cached objects, and the buffer pool is used to store cached objects that are evicted from the object pool. For a cached object, a credit score is determined based on the number of times the cached object is reused and the idle time. The number of times the cached object can be reused is determined based on the credit score. The credit score is positively correlated with the number of times it is reused and negatively correlated with the idle time. The number of times it can be reused is the maximum number of times the cached object is allowed to be reused in the buffer pool. In response to the failure to obtain a cached object from the object pool, the cached object with the largest difference between the reuse count quota and the reuse count is selected from the buffer pool as the target cached object, the target cached object is migrated to the object pool and returned.

[0005] Optionally, determining a credit score based on the number of times the cached object is reused and its idle time, and determining the reuse limit of the cached object based on the credit score, further includes: Get the number of times the cached object has been reused and its idle time; Credit scores are calculated based on the number of times the device is reused and the amount of time it is idle, with longer idle times resulting in lower credit scores. The reuse limit is calculated using credit scores and a preset base limit.

[0006] Optionally, calculating the credit score based on the number of reuses and idle time further includes: The attenuation factor is calculated based on the idle time and the preset attenuation coefficient. The attenuation factor decreases as the idle time increases. The product of the number of reuses and the decay factor is used as the credit score.

[0007] Optionally, calculating the reuse limit based on credit score and a preset base limit further includes: The product of the credit score and the preset base limit is rounded down to obtain the reuse limit.

[0008] Optionally, selecting the cache object with the largest difference between the reuse count and the reuse count from the buffer pool as the target cache object further includes: Iterate through each cached object in the buffer pool to obtain the reuse limit and reuse count for each cached object; Calculate the difference between the reuse limit and the reuse count for each cached object; The cache object corresponding to the maximum difference is determined as the target cache object.

[0009] Optionally, calculating the difference between the reuse limit and the reuse count for each cached object further includes: For each cached object, read its atomically stored reuse limit value and reuse count value; The difference between the reuse count limit and the reuse count is calculated and used as the difference for the cache object.

[0010] Optionally, maintaining the object pool and buffer pool further includes: When the number of cached objects in the object pool exceeds the object pool capacity threshold, the least used cached object is evicted from the object pool and stored in the buffer pool. In response to the target cache object being migrated out of the buffer pool, the target cache object is removed from the buffer pool.

[0011] Optionally, the method further includes: The buffer pool is scanned periodically. For each cached object, it is removed from the buffer pool when a preset recycling condition is met. The preset recycling condition includes exhausting the reuse limit or the idle time exceeding a preset threshold.

[0012] Optionally, the method further includes: Get system memory usage; Adjust at least one of the preset base quota, decay coefficient, and object pool capacity threshold based on the memory usage rate.

[0013] A second aspect of this disclosure provides a cache object scheduling system based on a host-isolated cache architecture, comprising: The object pool module is used to store active cached objects; The buffer pool module is used to store cached objects that have been evicted from the object pool module; The credit assessment module is used to determine a credit score based on the number of times a cached object is reused and the idle time, and to determine a reuse limit based on the credit score. The credit score is positively correlated with the number of reuses and negatively correlated with the idle time, and the reuse limit is the maximum number of times the cached object is allowed to be reused in the buffer pool. The scheduling module is used to respond to the fact that no cached object is obtained from the object pool module, select the cached object with the largest difference between the reuse count quota and the reuse count from the buffer pool module as the target cached object, migrate the target cached object to the object pool module and return.

[0014] As can be seen from the above technical solution, by maintaining a two-level structure of object pool and buffer pool, active objects are physically isolated from backup objects. The object pool prioritizes low-latency retrieval, while the buffer pool avoids immediately creating new objects. Based on this, a credit scoring and reuse limit mechanism based on reuse count and idle time is introduced: when a cached object is used up, its reuse count increases and its idle time is reset. Subsequent increases in idle time will lower its credit score through decay calculations, causing the score of long-term idle objects to drop rapidly. The credit score directly determines the reuse limit, i.e., the maximum number of times the object can be migrated and reused again in the backup area. When an object needs to be migrated from the buffer pool, the object with the largest difference between its reuse limit and current reuse count is selected. This setting ensures that objects with high historical reuse value and recent activity are prioritized for migration into the object pool for continued service, while the reuse limit of low-value objects will be exhausted rapidly during idle periods, and they will be directly reclaimed during scheduled scans due to limit exhaustion or idle timeout. Throughout the process, object migration and recycling are driven by the remaining reuse potential, which is quantified by the number of reuses and idle time. Migration decisions have clear dynamic quantitative basis, thereby improving the object reuse hit rate and significantly reducing the creation of new objects while ensuring low-latency acquisition of active areas. This reduces garbage collection frequency and memory fragmentation, while also enabling the timely removal of low-value objects and avoiding invalid memory occupation. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a cache object scheduling method based on a host-isolated cache architecture in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating the maintenance of the object pool and buffer pool in a cache object scheduling method based on a host-isolated cache architecture according to an embodiment of this disclosure; Figure 3 This is a flowchart illustrating a method for scheduling cache objects based on a host-isolated cache architecture, which involves determining a credit score based on the number of times the cache object is reused and its idle time, and then determining the reuse limit of the cache object based on the credit score. Figure 4 This is a flowchart illustrating the calculation of credit scores based on reuse counts and idle time in a cache object scheduling method based on a host-isolated cache architecture according to an embodiment of this disclosure. Figure 5This is a flowchart illustrating the process of selecting the cache object with the largest difference between the reuse count quota and the reuse count as the target cache object from the buffer pool, according to a cache object scheduling method based on a host-isolated cache architecture in an embodiment of this disclosure. Figure 6 This is a schematic diagram of a cache object scheduling system based on a host-isolated cache architecture in an embodiment of this disclosure. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not limiting, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without such specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0018] To address at least one of the above problems, this disclosure provides a cache object scheduling method based on a host-isolated caching architecture, such as... Figure 1 As shown, it includes: Step S100: Maintain the object pool and the buffer pool. The object pool is used to store active cached objects, and the buffer pool is used to store cached objects that are evicted from the object pool. Step S200: For a cached object, determine a credit score based on the number of times the cached object is reused and the idle time. Determine the number of times the cached object can be reused based on the credit score. The credit score is positively correlated with the number of times it is reused and negatively correlated with the idle time. The number of times it can be reused is the maximum number of times the cached object is allowed to be reused in the buffer pool. Step S300: In response to the failure to obtain a cached object from the object pool, select the cached object with the largest difference between the reuse count quota and the reuse count from the buffer pool as the target cached object, migrate the target cached object to the object pool and return the target cached object.

[0019] In this embodiment, by maintaining a two-tiered structure of object pool and buffer pool, active objects are physically isolated from backup objects. The object pool prioritizes low-latency retrieval, while the buffer pool avoids immediately creating new objects. A credit score is determined based on the reuse count and idle time of each cached object, and the reuse limit for that object is determined based on the credit score. The credit score is positively correlated with the reuse count and negatively correlated with idle time, causing the score of long-term idle objects to drop rapidly. When an object needs to be migrated from the buffer pool, the object with the largest difference between its reuse limit and current reuse count is selected. Therefore, object migration and recycling are entirely driven by the remaining reuse potential quantified by the reuse count and idle time. Migration decisions have clear dynamic quantitative basis, thereby improving the object reuse hit rate while ensuring low-latency retrieval in the active area, significantly reducing the creation of new objects, and thus reducing garbage collection frequency and memory fragmentation. Simultaneously, it enables the timely removal of low-value objects, avoiding invalid memory occupation.

[0020] The embodiments of this disclosure will now be described in detail.

[0021] In this embodiment, the application is applied to a distributed system employing a host-isolated caching architecture to manage cached objects generated during system operation. A host-isolated caching architecture means that each host maintains an independent cache instance, and cached data is only visible within that host. Task data on different hosts are isolated from each other, avoiding data cross-contamination. Under this architecture, multiple task modules run on each host, and each task module needs to use cached objects to store intermediate execution states. To reduce the system overhead caused by frequently creating and destroying cached objects, cached objects need to be reused and managed.

[0022] In step S100, an object pool and a buffer pool are maintained. The object pool is used to store active cached objects, and the buffer pool is used to store cached objects evicted from the object pool.

[0023] In this embodiment, the object pool and the buffer pool are two independent memory areas used to store cached objects at different lifecycle stages. The object pool stores active cached objects, which are cached objects that have been used recently and have a high probability of being used again in the future. The buffer pool stores cached objects evicted from the object pool. Although these objects have not been used recently, they still have a certain possibility of being reused. Storing them in the buffer pool can avoid immediate destruction, and when they are needed later, they can be directly retrieved from the buffer pool without recreating them.

[0024] It's important to note that the object pool and buffer pool are stored in a physically isolated manner, meaning they occupy different memory spaces and do not overlap. This physical isolation prevents objects with different lifecycles from being coexisting, reducing memory fragmentation. Furthermore, physical isolation allows for different management strategies for different pools, improving management flexibility and efficiency. Conversely, while logical isolation, dividing the same memory region into different logical partitions, is simple to implement, it cannot fundamentally prevent memory overlap between objects with different lifecycles, and memory fragmentation remains. This application uses physical isolation to store objects with different lifecycles in completely independent memory regions, resulting in more regular memory allocation and reclamation, effectively reducing memory fragmentation.

[0025] In an alternative embodiment, such as Figure 2 As shown, step S100 further includes: Step S101: When the number of cached objects in the object pool exceeds the object pool capacity threshold, the least used cached object is removed from the object pool and the removed cached object is stored in the buffer pool. Step S102: In response to migrating the target cache object out of the buffer pool, remove the target cache object from the buffer pool.

[0026] In this embodiment of the disclosure, the object pool has a fixed capacity threshold. When the number of cached objects in the object pool exceeds this capacity threshold, the system will perform an eviction operation, removing the least recently used cached object from the object pool. The least recently used cached object refers to the object in the object pool that was last used earliest.

[0027] It's important to note that the least recently used eviction policy is used because it effectively reflects the recent usage of cached objects. Least recently used objects typically have a low probability of being used again in the future; evicting them from the object pool frees up space for new, active objects, improving the overall efficiency of the object pool. This policy can be implemented using a doubly linked list. Each node in the doubly linked list stores a cached object and its last usage timestamp. When a new cached object needs to be added to the object pool, it is added to the tail of the list. When a cached object needs to be retrieved from the pool, it is removed from the list and then added to the tail. Thus, the object at the head of the list is the least recently used object. When the object pool is full and objects need to be evicted, only the head node needs to be removed. Operations on doubly linked lists require a mutex lock for protection because doubly linked lists are not thread-safe data structures. Under multi-threaded concurrent access, without a lock, the list structure may be corrupted. For example, while one thread is deleting a node from the head of a linked list, another thread might be adding a node to the tail. This could cause pointer errors in the linked list, leading to program crashes. Therefore, before performing any operation on a doubly linked list, a mutex lock must be acquired, and then released after the operation is complete. To reduce the performance impact of lock contention, fine-grained locks can be used, or the object pool can be sharded, with each shard using an independent lock. This allows operations on different shards to be executed in parallel, improving concurrency performance.

[0028] In this embodiment of the disclosure, evicted cached objects are stored in a buffer pool. Before being stored in the buffer pool, the system updates the object's credit score and reuse limit. The update is based on the object's current reuse count and idle time. After the update is completed, the object is added to the buffer pool.

[0029] This setting stores evicted objects in a buffer pool instead of immediately destroying them, preserving their reusability. When a cached object is needed later, if the object pool is empty, it can be retrieved from the buffer pool without being recreated, thus reducing system overhead. If these objects are destroyed immediately, they must be recreated when needed later, increasing garbage collection frequency and system overhead. Storing evicted objects in a buffer pool is a space-for-time tradeoff strategy that improves system performance within the limits of memory.

[0030] In this embodiment, when a target cached object is migrated from the buffer pool, the system removes the target cached object from the buffer pool. This removal operation is to prevent the object from being retrieved simultaneously by multiple callers, which could lead to data races and inconsistencies. Simultaneously, the removal operation also ensures that objects in the buffer pool are unused, preventing reuse.

[0031] It's important to note that if the target cached object isn't removed from the buffer pool, multiple threads might simultaneously select the same object as the target cached object, leading to data races and inconsistencies. Therefore, before migrating the target cached object to the object pool, it must first be removed from the buffer pool to ensure that the object can only be retrieved and used by one thread.

[0032] Step S200: For a cached object, determine a credit score based on the number of times the cached object is reused and the idle time. Determine the number of times the cached object can be reused based on the credit score. The credit score is positively correlated with the number of times it is reused and negatively correlated with the idle time. The number of times it can be reused is the maximum number of times the cached object is allowed to be reused in the buffer pool.

[0033] In this embodiment of the disclosure, a cache object refers to an object instance used in the system to store intermediate data, and each cache object has its own lifecycle. The reuse count of a cache object refers to the total number of times the object has been used since its creation; the reuse count increases each time it is used. The idle time of a cache object refers to the time interval between the last time the object was used and the current time; the longer the idle time, the lower the probability that the object will be used again.

[0034] It's important to note that both reuse counts and idle time are stored and updated using atomic operations. Atomic operations are uninterruptible operations that guarantee data consistency and correctness in multi-threaded concurrent access scenarios. Modern processors provide a series of atomic instructions, such as compare-and-swap, load-and-increment, and store-and-decrement instructions. These instructions guarantee atomicity at the hardware level, eliminating the need for software-level locking and thus offering high execution efficiency. In some programming languages, standard libraries provide wrappers for these atomic instructions, allowing developers to directly implement atomic operations. Without atomic operations, data races can occur in multi-threaded concurrent access scenarios. For example, when two threads simultaneously update the reuse count of the same cached object, one thread's update might be overwritten, leading to inaccurate reuse counts. Using atomic operations avoids this, ensuring data consistency and correctness.

[0035] In this embodiment, the credit score is an indicator used to quantitatively evaluate the reuse value of cached objects. The credit score is positively correlated with the number of reuses, meaning that the more times a cached object is reused, the higher its credit score, indicating that the object has greater historical reuse value and a higher probability of being used again in the future. The credit score is negatively correlated with idle time, meaning that the longer a cached object is idle, the lower its credit score, indicating that the object has lower current activity and a lower probability of being used again in the future.

[0036] This setting combines reuse counts and idle time to calculate a credit score, providing a more comprehensive and accurate assessment of the actual reuse value of cached objects. If only reuse counts are considered, objects historically frequently used but recently idle will receive higher scores, causing these low-value objects to remain in memory for extended periods, consuming system resources. Conversely, if only idle time is considered, objects historically rarely used but recently used will receive higher scores, leading to the premature reclamation of these high-potential objects and increased system overhead. Using a single dimension for evaluation fails to simultaneously consider historical value and current activity, resulting in inaccurate assessments. This disclosure combines these two dimensions to more accurately reflect the actual reuse value of cached objects, providing a reliable basis for subsequent management decisions.

[0037] In this embodiment, the reuse limit refers to the maximum number of times a cached object is allowed to be reused in the buffer pool. The reuse limit is determined based on a credit score; the higher the credit score, the larger the reuse limit, indicating that the object can be reused more times in the buffer pool and has a longer lifespan. Conversely, the lower the credit score, the smaller the reuse limit, indicating that the object can be reused fewer times in the buffer pool and has a shorter lifespan.

[0038] It's important to note that the reuse limit transforms the abstract credit score into a quantifiable survival credential, enabling differentiated lifecycle management of cached objects. High-value cached objects are given a higher reuse limit, allowing them to remain in the buffer pool longer and have more opportunities for reuse. Low-value cached objects are given a lower reuse limit, allowing them to be reclaimed more quickly, releasing system resources. If a fixed lifecycle management strategy is adopted, all objects have the same lifespan, making differentiated management based on actual object value impossible. This results in high-value objects being prematurely reclaimed, while low-value objects remain in memory for extended periods. This disclosure, by employing a reuse limit mechanism, achieves dynamic adjustment of object lifecycles, better balancing reuse rate and memory usage.

[0039] In an alternative embodiment, such as Figure 3As shown, a credit score is determined based on the number of times the cached object is reused and its idle time. Further determining the reuse limit for the cached object based on the credit score includes: Step S201: Obtain the reuse count and idle time of the cached object; Step S202: Calculate the credit score based on the number of reuses and the idle time, where the longer the idle time, the lower the credit score; Step S203: Calculate the reuse limit using credit score and preset basic limit.

[0040] In this embodiment, obtaining the reuse count and idle time of a cached object is a prerequisite for calculating a credit score. The reuse count is stored in the cached object's metadata, and the system automatically updates this value each time the cached object is used. The idle time can be obtained by subtracting the timestamp of the last use of the cached object from the current time. The timestamp of the last use is also stored in the cached object's metadata, and the system automatically updates this value to the current time each time the cached object is used.

[0041] It's important to note that both the reuse count and the last usage timestamp are stored and updated using atomic operations. In multi-threaded concurrent access scenarios, atomic operations ensure the consistency and correctness of these values. For example, when multiple threads use the same cached object simultaneously, each thread's usage is correctly recorded, the reuse count is accurately incremented, and the last usage timestamp is updated to the latest time. Without atomic operations, data loss or errors could occur, leading to inaccurate credit score calculations and impacting the entire system's operation.

[0042] In this embodiment, when calculating a credit score based on reuse count and idle time, the combined effect of two factors needs to be considered. Reuse count reflects the historical reuse value of the cached object, while idle time reflects its current activity level. By combining these two factors, a score that accurately reflects the actual reuse value of the cached object can be obtained.

[0043] It should be noted that there are multiple ways to calculate credit scores, as long as they meet the conditions of being positively correlated with the number of reuses and negatively correlated with idle time. For example, different function forms such as linear functions, exponential functions, and power functions can be used to calculate credit scores. Different function forms are suitable for different business scenarios, and the appropriate form can be selected based on actual needs. The characteristic of linear functions is that they are simple to calculate, and the score changes linearly with the number of reuses and idle time. The characteristic of exponential functions is that they can quickly reduce the score of long-term idle objects, which conforms to the pattern of most business scenarios. The characteristic of power functions is that the decay rate is between that of linear and exponential functions, and the power can be adjusted to adapt to different scenarios. This disclosure uses the form of exponential functions to calculate credit scores because it can better fit the trend of the reuse probability of cached objects changing with idle time.

[0044] In this embodiment, when calculating the reuse limit using the credit score and a preset base limit, the credit score is multiplied by the preset base limit, and then rounded down to obtain the reuse limit. The preset base limit is a pre-set constant used to adjust the overall size of the reuse limit. The larger the base limit, the larger the reuse limit corresponding to the same credit score, and the longer the cached object can survive in the buffer pool. The smaller the base limit, the smaller the reuse limit corresponding to the same credit score, and the shorter the survival time of the cached object in the buffer pool.

[0045] It should be noted that the preset base limit can be adjusted based on the system's memory usage and business needs. When system memory is sufficient, the base limit can be increased appropriately to keep more cached objects in the buffer pool, improving the reuse hit rate. When system memory is tight, the base limit can be decreased appropriately to accelerate the reclamation of low-value objects and release system resources. The initial value of the preset base limit can be determined through experience or testing, and then dynamically adjusted according to the actual situation during system operation.

[0046] Optionally, such as Figure 4 As shown, calculating a credit score based on the number of reuses and idle time further includes: Step S211: Calculate the attenuation factor based on the idle time and the preset attenuation coefficient. The attenuation factor decreases as the idle time increases. Step S221: The product of the number of reuses and the attenuation factor is used as the credit score.

[0047] In this embodiment of the disclosure, the decay factor is an indicator used to quantify the impact of idle time on credit scores. The decay factor decreases as the idle time increases; that is, the longer the idle time, the smaller the decay factor, and the lower the credit score. The decay factor can be calculated in various ways, such as exponential decay, linear decay, power-law decay, etc.

[0048] Specifically, exponential decay is a commonly used decay method. Its characteristic is that the decay factor decreases slowly when the idle time is short, but drops rapidly after the idle time exceeds a certain threshold. This characteristic aligns with actual business patterns, because objects that have only recently been idle still have a high probability of being reused, while the probability of reuse for long-term idle objects drops sharply. Linear decay is characterized by a uniform decay rate, suitable for scenarios where the reuse probability decreases linearly with idle time. Power-law decay is characterized by a rapid initial decay rate followed by a slower rate, suitable for scenarios where the reuse probability decreases rapidly in the initial stage but then remains at a low level. This disclosure uses exponential decay to calculate the decay factor because it better reflects the changing trend of cached object reuse probability in most business scenarios.

[0049] In this embodiment of the disclosure, the preset attenuation coefficient is a pre-set constant used to adjust the rate at which the attenuation factor changes with idle time. The larger the attenuation coefficient, the faster the attenuation factor decreases with increasing idle time, and the greater the impact of idle time on credit score; the smaller the attenuation coefficient, the slower the attenuation factor decreases with increasing idle time, and the smaller the impact of idle time on credit score.

[0050] The preset decay coefficient can be adjusted according to the characteristics of the business scenario. For scenarios with short object lifecycles and high reuse frequency, the decay coefficient can be appropriately increased to allow long-term idle objects to be reclaimed more quickly. For scenarios with long object lifecycles and low reuse frequency, the decay coefficient can be appropriately decreased to allow objects to remain in the buffer pool for a longer period and have more opportunities to be reused. The initial value of the preset decay coefficient can be determined through experience or testing, and then dynamically adjusted according to the actual situation during system operation. For example, different business loads can be simulated in a test environment, and the value of the decay coefficient can be adjusted to observe the system's performance, such as reuse hit rate, garbage collection frequency, and memory utilization, thereby finding the most suitable decay coefficient value for the current business scenario.

[0051] In this embodiment, the product of the number of reuses and the decay factor is used as the credit score. This calculation method is simple and efficient, and accurately reflects the combined impact of the number of reuses and idle time on the credit score. The greater the number of reuses, the higher the credit score; the smaller the decay factor, the lower the credit score.

[0052] It's important to note that this calculation method uses the number of reuses as a basis and then adjusts the weights through a decay factor, ensuring that recently active objects receive higher scores, while the scores of long-inactive objects drop rapidly. This approach considers both the object's historical contribution and its current activity level, thus more accurately reflecting the object's actual reuse value.

[0053] Optionally, calculating the reuse limit using credit score and preset basic limit further includes: rounding down the product of credit score and preset basic limit to obtain the reuse limit.

[0054] Rounding down means taking the largest integer not greater than the product. For example, if the product of the credit score and the preset basic limit is 17.8, then the reuse limit after rounding down is 17.

[0055] Using rounding down avoids decimal reuse counts, as the reuse count should be an integer, and therefore the reuse count should also be an integer. Rounding down also appropriately reduces the reuse count, speeding up the reclamation of low-value objects and preventing memory waste. If rounding were used, some low-value objects might receive higher reuse counts, remaining in memory for extended periods and consuming system resources. Rounding down is more conservative and better controls memory usage.

[0056] In this embodiment, the preset base limit can be set according to actual needs. For example, the preset base limit can be set to different values ​​such as 5, 10, and 15. The size of the base limit affects the overall level of the reuse count limit, and thus affects the survival time of cached objects in the buffer pool. The larger the base limit, the larger the reuse count limit corresponding to the same credit score, and the longer the object can survive in the buffer pool. The smaller the base limit, the smaller the reuse count limit corresponding to the same credit score, and the shorter the survival time of the object in the buffer pool.

[0057] It's important to note that the optimal value for the preset base limit needs to be determined through actual testing and optimization. Developers can first set an initial base limit based on the system's memory capacity and workload, and then adjust it according to the system's performance. If the system's memory utilization is high, the base limit can be appropriately reduced; if the system's reuse hit rate is low, the base limit can be appropriately increased. Furthermore, the base limit can also be dynamically adjusted along with other parameters to adapt to different workloads and memory environments.

[0058] In step S300, in response to the failure to obtain a cached object from the object pool, the cached object with the largest difference between the reuse count quota and the reuse count is selected from the buffer pool as the target cached object, the target cached object is migrated to the object pool and returned.

[0059] In this embodiment of the disclosure, when the system needs to obtain a cached object, it first attempts to obtain it from the object pool. If there is an available cached object in the object pool, the object is directly returned to the caller; if there is no available cached object in the object pool, the system will select a suitable cached object from the buffer pool for migration.

[0060] When selecting cached objects from the buffer pool, the object with the largest difference between its reuse count and reuse count is chosen as the target cached object. The difference represents the remaining reuse potential of the cached object; the larger the difference, the more unused reuse opportunities the object has, and the higher the probability of it being used again in the future. Therefore, selecting the object with the largest difference for migration maximizes the reuse value of each cached object and improves the overall reuse hit rate.

[0061] Various strategies exist for selecting objects from the backup area in related technologies. Some strategies select objects with the highest total reuse count. The problem with this strategy is that objects with a high total reuse count may have been idle for a long time and have a low probability of being reused in the future, while objects with a low total reuse count but recent activity may be ignored. Some strategies select objects with the shortest idle time. The problem with this strategy is that objects with a short idle time may have low historical reuse value and have only been used occasionally, with a low probability of being reused in the future. Some strategies use a first-in, first-out (FIFO) approach. This strategy completely ignores the value differences of objects and often removes objects that have just been phased out but still have the potential for reuse. Some strategies use random selection, but the effect of this strategy is unstable and cannot guarantee the reuse hit rate. The difference selection strategy disclosed in this paper considers both the historical reuse value and the current usage of an object. The reuse count reflects the comprehensive value of the object, and the current reuse count reflects the degree of use of the object. The difference between the two accurately reflects the remaining reuse potential of the object. This strategy can prioritize objects with high comprehensive value that have not yet been fully used, thereby improving the overall reuse hit rate, reducing the creation of new objects, and reducing garbage collection overhead.

[0062] In this embodiment, when a target cached object is selected, the system removes the object from the buffer pool, migrates it to the object pool, and finally returns it to the caller. Removing the target cached object from the buffer pool prevents multiple callers from simultaneously retrieving the object, which could lead to data races and inconsistencies. Simultaneously, the removal operation ensures that objects in the buffer pool are unused, preventing reuse.

[0063] It should be noted that some methods for cached object reuse management in related technologies employ a single object pool structure, storing all cached objects in the same pool and using a first-in, first-out (FIFO) or least recently used (LRU) strategy for eviction. This method cannot distinguish between objects with different reuse values, leading to the early reclamation of high-value objects and the long-term residence of low-value objects in memory. Other methods use a two-level pool structure, dividing objects into an active area and a backup area. However, when migrating objects from the backup area to the active area, random selection or a single-dimensional selection strategy is typically used. For example, some methods select objects based on total reuse counts, failing to distinguish between historically high-frequency but recently idle objects and historically low-frequency but recently active objects; others select objects based on idle time, failing to distinguish between objects with different historical reuse values; still others use a FIFO strategy, completely ignoring the value differences between objects. The various features in related technologies exist in a scattered manner and are not combined in the specific way described in this disclosure to achieve fine-grained and intelligent management of cached objects. This disclosure organically combines a two-level physical isolation architecture, a two-dimensional credit scoring mechanism, a reuse limit mechanism, and a difference maximization migration strategy to form a complete closed-loop system. This system can reduce garbage collection overhead while avoiding low-value objects from residing in memory for a long time, achieving good technical results.

[0064] In some alternative embodiments, such as Figure 5 As shown, selecting the cache object with the largest difference between the reuse count and the reuse count from the buffer pool as the target cache object further includes: Step S301: Traverse each cached object in the buffer pool and obtain the reuse limit and reuse count of each cached object; Step S302: Calculate the difference between the reuse limit and the reuse count for each cached object; Step S303: Determine the cache object corresponding to the maximum difference as the target cache object.

[0065] Specifically, when a target cached object needs to be selected from the buffer pool, the system iterates through all cached objects in the buffer pool. The iteration can be done in various ways, such as sequential or reverse order. During the iteration, the system obtains the reuse limit and reuse count for each cached object.

[0066] The cached objects in the buffer pool are stored in a traversable data structure, such as a linked list, array, or map. Different data structures are suitable for different scenarios and can be selected according to actual needs. For example, a map structure supports fast lookup and deletion operations, suitable for scenarios requiring frequent object migration and recycling. A linked list structure supports fast insertion and deletion operations, suitable for scenarios requiring frequent object addition and removal. An array structure supports fast random access, suitable for scenarios with many traversal operations. This application uses a map structure as the storage structure for the buffer pool because it can well support various operations of the buffer pool and has high execution efficiency.

[0067] In this embodiment, the difference between the reuse limit and the number of reuses for each cached object is calculated. Specifically, the difference is calculated by subtracting the number of reuses from the reuse limit. The larger the difference, the greater the remaining reuse potential of the cached object, and the higher the probability of it being reused in the future.

[0068] It's important to note that the difference is calculated based on the current reuse limit and reuse count. Since the reuse limit decreases with increasing idle time, and the reuse count increases with object usage, the difference is a dynamically changing value. Each time a target cached object is selected, the difference for all objects needs to be recalculated to ensure that the object with the highest reuse potential is selected. Using historical differences might lead to the selection of objects that have lost their reuse potential, affecting the reuse hit rate.

[0069] Optionally, calculating the difference between the reuse limit and the reuse count for each cache object further includes: for each cache object, reading its atomically stored reuse limit value and reuse count value; calculating the reuse limit value minus the reuse count value as the difference for that cache object.

[0070] Specifically, atomic storage refers to storing data in memory using atomic operations. In multi-threaded concurrent access scenarios, atomic storage guarantees data consistency and correctness. The result of subtracting the reuse count from the reuse limit reflects the remaining reuse potential of the cached object. A larger value indicates that the object has more unused reuse opportunities and a higher probability of being used again in the future.

[0071] Both the reuse count limit and the reuse count are stored in the metadata of the cached object and are of atomic integer type. Atomic integer type provides a series of atomic operation methods, such as read, increment, decrement, compare and swap, etc. These methods ensure that data will not be raced or corrupted under multi-threaded concurrent access. Without atomic storage, corrupted data might be read under multi-threaded concurrent access. For example, while one thread is updating the reuse count limit, another thread might read an incorrect value that is only partially updated, leading to an incorrect difference calculation and selection of the wrong target cached object. Using atomic storage avoids this situation, guaranteeing data consistency and correctness.

[0072] It's important to note that calculating the difference is a simple arithmetic operation with very high computational efficiency. Even with a large number of cached objects in the buffer pool, iterating through all objects and calculating the difference will not consume significant system resources and will not have a noticeable impact on system performance. Furthermore, the iteration and calculation process can be executed asynchronously in the background without blocking the normal operation of business threads.

[0073] In this embodiment, the cached object corresponding to the maximum difference is determined as the target cached object. If multiple cached objects have the same maximum difference, then any one of them can be selected as the target cached object. For example, the object that is traversed first can be selected, or the object with the shortest idle time can be selected.

[0074] It's important to note that when multiple objects have the same difference in value, choosing different objects has little impact on the overall system performance. This is because these objects have roughly the same remaining reuse potential, and choosing any one will result in good reuse performance. To simplify implementation, you can choose the object that is traversed first. If further optimization is needed, you can choose the object with the shortest idle time, as it has higher current activity and a slightly higher probability of being reused in the future.

[0075] In another optional embodiment, the method further includes: periodically scanning the buffer pool, and for each cached object, removing it from the buffer pool when a preset recycling condition is met, the preset recycling condition including exhausting the reuse count or the idle time exceeding a preset threshold.

[0076] In this embodiment, the system periodically scans the buffer pool to check whether each cached object meets the preset reclamation conditions. The periodic scanning cycle can be set according to actual needs, such as scanning every 10 minutes, every 15 minutes, or every 30 minutes. The length of the scanning cycle affects the timeliness of cached object reclamation and system overhead. The shorter the scanning cycle, the more timely the reclamation, but the greater the system overhead; the longer the scanning cycle, the smaller the system overhead, but the reclamation may be less timely.

[0077] It's important to note that the scheduled scanning operation runs asynchronously in the background and will not block the normal operation of business threads. The background scanning thread is periodically woken up to traverse all objects in the buffer pool and check if they meet the reclamation criteria. For objects that meet the reclamation criteria, the system will remove them from the buffer pool and release the memory resources they occupy. The priority of the background scanning thread can be set to a low level to avoid affecting the operation of business threads. Furthermore, the scanning process can be performed in batches to avoid system lag caused by traversing a large number of objects at once.

[0078] In this embodiment of the disclosure, the preset reclamation conditions include exhausting the reuse count or exceeding a preset threshold. Exhausting the reuse count means that the reuse count of the cached object is less than or equal to 0, indicating that the object has no remaining reuse opportunities and should be reclaimed. Exceeding the preset threshold means that the idle time of the cached object exceeds the preset maximum allowed idle time, indicating that the object has not been used for a long time and the probability of it being used again in the future is extremely low, and it should be reclaimed.

[0079] The above settings, by combining two reclamation conditions, can more comprehensively cover objects that need to be reclaimed. The condition of exhausting reuse limits primarily targets objects that, although idle for a short time, have a low credit score and no reuse value. The condition of exceeding a preset threshold for idle time primarily targets objects that, although idle for a long time, have not been used and have a very low probability of being used again in the future. By using both conditions, it ensures that all objects without reuse value are reclaimed in a timely manner, freeing up system resources. Using only one condition might result in some objects without reuse value remaining resident in memory for an extended period, consuming system resources.

[0080] In this embodiment, the preset threshold can be set according to actual needs. For example, the maximum allowed idle time can be set to 12 hours, 24 hours, 48 ​​hours, etc. The size of the preset threshold affects the maximum lifespan of cached objects in the buffer pool. The larger the threshold, the longer the object can survive, but the larger the memory usage; the smaller the threshold, the shorter the object can survive, but the smaller the memory usage.

[0081] The optimal value for the preset threshold needs to be adjusted based on the actual business scenario. For scenarios where data updates are frequent and object validity is short, the maximum allowable idle time can be appropriately shortened; for scenarios where data updates are slow and object validity is long, the maximum allowable idle time can be appropriately extended. Furthermore, the preset threshold can also be dynamically adjusted along with other parameters to adapt to different business loads and memory environments.

[0082] In another optional embodiment, the method further includes: obtaining the system memory usage rate; and adjusting at least one of a preset base quota, a decay coefficient, and an object pool capacity threshold based on the memory usage rate.

[0083] In this embodiment of the disclosure, the system obtains the current system memory usage rate in real time. The system memory usage rate refers to the proportion of memory currently used by the system to the total memory. The memory usage rate can reflect the memory pressure of the system. The higher the memory usage rate, the greater the memory pressure of the system, and the more necessary it is to speed up the reclamation of cached objects and release more memory resources.

[0084] There are several ways to obtain system memory usage. Different operating systems and programming languages ​​provide different interfaces and methods. The system can obtain the current memory usage by calling the interfaces provided by the operating system and then calculate the memory usage rate. For example, in some operating systems, memory usage information can be obtained by reading specific system files. In some programming languages, the standard library provides functions for obtaining system memory information. The operation of obtaining system memory usage should be as lightweight as possible to avoid consuming excessive system resources. At the same time, the frequency of obtaining it should also be appropriate to avoid increasing system overhead due to excessively frequent access.

[0085] In this embodiment, at least one of a preset base limit, a decay coefficient, and an object pool capacity threshold is adjusted based on memory usage. The adjustment principle is as follows: when memory usage is high, the preset base limit is reduced, the decay coefficient is increased, and the object pool capacity threshold is decreased to accelerate the reclamation of low-value objects and release memory resources. When memory usage is low, the preset base limit is increased, the decay coefficient is decreased, and the object pool capacity threshold is increased to improve object reuse hit rate and reduce system overhead.

[0086] It's important to note that parameter adjustment is a dynamic process; the system adjusts these parameters in real time based on changes in memory usage. This dynamic parameter adjustment mechanism allows the system to adapt to different workloads and memory environments, maximizing system performance while ensuring stability. Using fixed parameters would prevent full utilization of memory to improve reuse hit rate when memory is plentiful, and would hinder timely object reclamation when memory is scarce, potentially leading to memory overflow. The dynamic parameter adjustment mechanism automatically adjusts parameters based on the system's actual situation, achieving a balance between performance and stability.

[0087] Parameter adjustments can be categorized into multiple levels, with different parameter values ​​corresponding to different memory usage ranges. For example, memory usage can be divided into three ranges: low, medium, and high. When memory usage is in the low range, a larger preset base allowance, a smaller decay coefficient, and a larger object pool capacity threshold are used. When memory usage is in the medium range, a medium preset base allowance, a medium decay coefficient, and a medium object pool capacity threshold are used. When memory usage is in the high range, a smaller preset base allowance, a larger decay coefficient, and a smaller object pool capacity threshold are used. This tiered adjustment method is simple to implement and can meet the needs of most scenarios.

[0088] The frequency of parameter adjustments should be appropriate to avoid system instability caused by excessive adjustments. For example, an adjustment interval can be set, allowing parameters to be adjusted only at regular intervals. Alternatively, an adjustment threshold can be set, requiring parameter adjustments only when memory usage changes by a certain margin, thus avoiding frequent adjustments due to small fluctuations in memory usage and ensuring system stability.

[0089] In this embodiment, the cached object scheduling system can be implemented through multiple functional modules. These modules cooperate to jointly complete the scheduling and management of cached objects. The object pool module stores active cached objects and implements a least-used eviction policy. The buffer pool module stores cached objects evicted from the object pool module and provides functions for adding, removing, and traversing objects. The credit evaluation module determines a credit score based on the number of times a cached object is reused and its idle time, and determines a reuse limit based on the credit score. The scheduling module responds to cached object retrieval requests, retrieves a suitable cached object from the object pool module or the buffer pool module, and returns it to the caller.

[0090] It's important to note that the modules interact through interfaces, and the internal implementation details of each module are transparent to others. This modular design approach gives the system excellent scalability and maintainability. If the implementation of a module needs to be modified, only the interface needs to remain unchanged; this will not affect the normal operation of other modules. For example, if the storage structure of the buffer pool needs to be changed from one concurrency-safe mapping to another, only the implementation of the buffer pool module needs to be modified; the code of the object pool module, credit evaluation module, and scheduling module does not need to be modified.

[0091] In this embodiment, cleaning up cached objects is also a crucial step. When a cached object is returned to the object pool or migrated from the buffer pool to the object pool, it needs to be cleaned up to delete old data and prevent dirty data residue. Dirty data residue may cause errors in modules that subsequently use the object, affecting the correctness of the system.

[0092] It's important to note that the cleanup operation for cached objects can be performed with varying intensity depending on the pool in which the object resides. For objects in the object pool, since they are about to be used, a full cleanup is necessary, deleting all key-value pairs and resetting the internal data structure. For objects in the buffer pool, since they will not be used for the time being, incremental cleanup can be performed, deleting only key-value pairs that have not been accessed for a certain period. This reduces unnecessary memory copying operations and improves system performance. At the same time, the cleanup operation should be as efficient as possible to avoid consuming excessive system resources.

[0093] The storage of metadata for cached objects is also an issue that needs to be considered. The metadata of cached objects includes information such as the number of times they have been reused, the timestamp of the last use, and the reuse limit. This metadata needs to be stored together with the cached object itself so that the system can retrieve and update this information at any time.

[0094] Specifically, there are several ways to store metadata. One way is to store the metadata as a member variable of the cache object, along with the cache object itself. The advantage of this method is fast access speed and no need for an additional mapping table. Another way is to store the metadata in a separate mapping table, where the key is the unique identifier of the cache object and the value is the metadata. The advantage of this method is that it does not require modification of the cache object's structure and has better compatibility. In this embodiment, the first method is used, storing the metadata as a member variable of the cache object, which achieves higher access speed. If compatibility with existing cache object structures is required, the second method can also be used.

[0095] For example, the technical solution of this application will be described in detail below with reference to a specific application scenario. In this example, the system adopts a host-isolated caching architecture to manage module cache objects during the cluster deployment process. Each module cache object is used to store intermediate state data during the execution of a single module. The system's current memory utilization rate is 65%, indicating that it is in normal operating condition.

[0096] During system initialization, the various modules of the cached object scheduling system are created. The capacity threshold of the object pool module is set to 50, and the buffer pool module is initially empty. The preset base quota is set to 10, and the preset decay coefficient is set to 0.5. The periodic scanning cycle of the buffer pool is set to 15 minutes, and the maximum allowed idle time is set to 24 hours.

[0097] When the first pipeline task begins execution, the system needs to allocate a module cache object for each module. The first module calls the scheduler module's retrieval interface to request a module cache object. The scheduler module first checks the object pool module and finds it empty, then checks the buffer pool module and finds it empty as well. Therefore, the scheduler module creates a new module cache object and returns it to the first module for use.

[0098] After the first module finishes using the cached object, it calls the return interface of the scheduling module to return the object. The scheduling module then stores the object in the object pool module. At this point, the number of objects in the object pool is 1, which is less than the capacity threshold of 50, so no eviction operation is required.

[0099] As the pipeline continues to execute, more and more modules need to use the module cache objects. When the number of objects in the object pool reaches 50, the object pool is full. At this point, the 51st module calls the retrieval interface, and the scheduling module retrieves the least used object from the object pool module and returns it to the 51st module. After the 51st module finishes using the object, it returns it to the object pool module. Since the object pool is full, the scheduling module evicts the least used object at the head of the object pool and stores it in the buffer pool module. Before the evictened object is stored in the buffer pool module, the credit assessment module calculates a credit score and reuse limit based on the object's reuse count and idle time. Assuming the object's reuse count is 2 and idle time is 0 hours, then the decay factor is the natural constant -0.5 multiplied by 0, which equals 1. The credit score is 2 multiplied by 1, which equals 2. The reuse limit is 2 multiplied by 10 and rounded down, which equals 20.

[0100] When the 52nd module calls the retrieve interface, the same process occurs. Another object at the head of the object pool is evicted and stored in the buffer pool module. This continues until the first pipeline task completes, at which point the buffer pool module has stored 50 evicted module cache objects.

[0101] Fifteen minutes later, the system triggers a scheduled scan of the buffer pool. A background scanning thread iterates through all objects in the buffer pool, checking if they meet the reclamation criteria. At this point, each object has been idle for 15 minutes, or 0.25 hours. The decay factor is the natural constant raised to the power of -0.5 multiplied by 0.25, approximately 0.8825. The credit score is 2 multiplied by 0.8825, approximately 1.765. The reuse count is 1.765 multiplied by 10, rounded down, equal to 17. Since all objects have a reuse count greater than 0 and idle times less than 24 hours, no objects are reclaimed.

[0102] When the second pipeline task begins execution, the first module calls the retrieval interface again. The scheduling module first checks the object pool module and finds it to be full. Therefore, the scheduling module removes an object from the head of the object pool and adds it to the buffer pool module. At this point, the object pool is empty, and the scheduling module needs to select an object from the buffer pool module for migration.

[0103] The scheduling module iterates through all objects in the buffer pool, calculating the difference between the reuse limit and the reuse count for each object. For the 50 previously stored objects, their reuse limit is 17 and their reuse count is 2, resulting in a difference of 15. For the object that was just evicted, its reuse limit is 20 and its reuse count is 2, resulting in a difference of 18. Therefore, the scheduling module selects the object with the largest difference as the target cache object, removes it from the buffer pool module, migrates it to the object pool module, and then returns it to the first module for use.

[0104] After the first module finishes using the object, it returns it to the end of the object pool module. As the second pipeline task continues to execute, more modules will call the retrieval interface, and the scheduling module will continuously select the most reusable objects from the buffer pool module for migration.

[0105] 24 hours later, the system triggered another periodic scan of the buffer pool. At this point, objects that were discarded in the first pipeline task and not reused in the second pipeline task had been idle for 24 hours. The decay factor is 0.5 × 24 (the natural constant), approximately 1.523 × 10⁻⁵. The credit score is 2 × 1.523 × 10⁻⁵, approximately 3.046 × 10⁻⁵. The reuse limit is 3.046 × 10⁻⁵ × 10 rounded down, equal to 0. These objects met the criteria for reclaiming the reuse limit and were therefore removed from the buffer pool module, releasing the memory resources they occupied.

[0106] Throughout the process, the system monitors memory usage in real time. If memory usage rises above 80%, the system automatically adjusts the preset base limit to 5, the decay factor to 0.8, and the object pool capacity threshold to 20. This accelerates the reclamation of low-value objects and frees up more memory resources. If memory usage drops below 50%, the system automatically adjusts the preset base limit to 15, the decay factor to 0.3, and the object pool capacity threshold to 100. This improves object reuse hit rate and reduces system overhead.

[0107] Those skilled in the art will understand that the technical solution of this disclosure is not only applicable to module cache object management in cluster deployment scenarios, but also to other distributed system scenarios that require cache object reuse management. For example, in a microservice architecture, the technical solution of this application can be used to manage request cache objects during service calls; in a big data processing system, the technical solution of this application can be used to manage intermediate result cache objects during data processing. The scope of protection of this disclosure is not limited to the specific application scenarios mentioned above, and any equivalent substitutions and modifications made based on the technical concept of this application should be covered within the scope of protection of this application.

[0108] Another aspect of this disclosure provides a cache object scheduling system based on a host-isolated caching architecture, such as... Figure 6 As shown, it includes: Object pool module 10 is used to store active cached objects; Buffer pool module 20 is used to store cached objects evicted from the object pool module; The credit assessment module 30 is used to determine a credit score based on the number of times the cached object is reused and the idle time, and to determine the reuse limit based on the credit score. The credit score is positively correlated with the number of reuses and negatively correlated with the idle time. The reuse limit is the maximum number of times the cached object is allowed to be reused in the buffer pool. The scheduling module 40 is used to respond to the fact that no cached object is obtained from the object pool module, select the cached object with the largest difference between the reuse count quota and the reuse count from the buffer pool module as the target cached object, migrate the target cached object to the object pool module and return.

[0109] It should be noted that the specific embodiments of the cache object scheduling method executed by the cache object scheduling system based on the host-isolated cache architecture of this disclosure have been described in detail in the above method process, and will not be repeated here.

[0110] In the embodiments of this disclosure, a two-tier structure of object pool and buffer pool is maintained to physically isolate active objects from backup objects. The object pool prioritizes low-latency retrieval, while the buffer pool avoids immediately creating new objects. Based on this, a credit scoring and reuse limit mechanism based on reuse count and idle time is introduced: when a cached object is used up, its reuse count increases and its idle time is reset. Subsequent increases in idle time will decrease its credit score through decay calculations, causing the score of long-term idle objects to drop rapidly. The credit score directly determines the reuse limit, i.e., the maximum number of times the object can be migrated and reused again in the backup area. When an object needs to be migrated from the buffer pool, the object with the largest difference between its reuse limit and current reuse count is selected. This setting ensures that objects with high historical reuse value and recent activity are prioritized for migration into the object pool for continued service, while the reuse limit of low-value objects will be rapidly exhausted during idle periods and will be directly reclaimed during scheduled scans due to limit exhaustion or idle timeout. Throughout the process, object migration and recycling are driven by the remaining reuse potential, which is quantified by the number of reuses and idle time. Migration decisions have clear dynamic quantitative basis, thereby improving the object reuse hit rate and significantly reducing the creation of new objects while ensuring low-latency acquisition of active areas. This reduces garbage collection frequency and memory fragmentation, while also enabling the timely removal of low-value objects and avoiding invalid memory occupation.

[0111] Those skilled in the art will understand that the above description is merely a preferred embodiment of this disclosure and is not intended to limit the scope of protection of this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure. Further details are omitted here.

Claims

1. A cache object scheduling method based on a host-isolated caching architecture, characterized in that, include: Maintain an object pool and a buffer pool, wherein the object pool is used to store active cached objects and the buffer pool is used to store cached objects evicted from the object pool; For the cached object, a credit score is determined based on the number of times the cached object is reused and the idle time. The number of times the cached object can be reused is determined based on the credit score. The credit score is positively correlated with the number of times it is reused and negatively correlated with the idle time. The number of times it can be reused is the maximum number of times the cached object is allowed to be reused in the buffer pool. In response to the failure to obtain a cached object from the object pool, the cached object with the largest difference between the reuse count quota and the reuse count is selected from the buffer pool as the target cached object, the target cached object is migrated to the object pool and the target cached object is returned.

2. The method according to claim 1, characterized in that, The step of determining a credit score based on the reuse count and idle time of the cached object, and determining the reuse count limit of the cached object based on the credit score, further includes: Get the number of times the cached object has been reused and its idle time; A credit score is calculated based on the number of reuses and the idle time, wherein the longer the idle time, the lower the credit score; The reuse limit is calculated using the credit score and the preset basic limit.

3. The method according to claim 2, characterized in that, The calculation of the credit score based on the number of reuses and the idle time further includes: The attenuation factor is calculated based on the idle time and the preset attenuation coefficient, and the attenuation factor decreases as the idle time increases; The product of the number of reuses and the attenuation factor is used as the credit score.

4. The method according to claim 2, characterized in that, The step of calculating the reuse limit using the credit score and the preset basic limit further includes: The product of the credit score and the preset base limit is rounded down to obtain the reuse limit.

5. The method according to claim 1, characterized in that, The step of selecting the cache object with the largest difference between the reuse count limit and the reuse count from the buffer pool as the target cache object further includes: Iterate through each cached object in the buffer pool to obtain the reuse limit and reuse count of each cached object; Calculate the difference between the reuse limit and the reuse count for each cached object; The cache object corresponding to the maximum difference is determined as the target cache object.

6. The method according to claim 5, characterized in that, The calculation of the difference between the reuse limit and the reuse count for each cached object further includes: For each cached object, read its atomically stored reuse limit value and reuse count value; The result of subtracting the reuse count value from the reuse count value is used as the difference for the cache object.

7. The method according to claim 1, characterized in that, The maintenance object pool and buffer pool further include: When the number of cached objects in the object pool exceeds the object pool capacity threshold, the least used cached object is removed from the object pool and stored in the buffer pool. In response to the target cache object being migrated out of the buffer pool, the target cache object is removed from the buffer pool.

8. The method according to claim 1, characterized in that, Also includes: The buffer pool is scanned periodically. For each cached object, it is removed from the buffer pool when a preset recycling condition is met. The preset recycling condition includes exhausting the reuse limit or the idle time exceeding a preset threshold.

9. The method according to claim 8, characterized in that, Also includes: Get system memory usage; Adjust at least one of the preset base quota, decay coefficient, and object pool capacity threshold based on the memory usage rate.

10. A cache object scheduling system based on a host-isolated cache architecture, characterized in that, include: The object pool module is used to store active cached objects; The buffer pool module is used to store cached objects that have been evicted from the object pool module; The credit assessment module is used to determine a credit score based on the number of times a cached object is reused and the idle time, and to determine a reuse limit based on the credit score. The credit score is positively correlated with the number of reuses and negatively correlated with the idle time, and the reuse limit is the maximum number of times the cached object is allowed to be reused in the buffer pool. The scheduling module is used to respond to the fact that no cached object is obtained from the object pool module, select the cached object with the largest difference between the reuse count quota and the reuse count from the buffer pool module as the target cached object, migrate the target cached object to the object pool module and return.

Images