Cache control method and system, storage medium and electronic equipment
By implementing a management mechanism that writes a list of data blocks into the cache service and using token bucket control, the problem of memory resource inflation in log-type cache services has been solved, enabling the rational use of cache resources, avoiding system crashes and performance bottlenecks, and improving system stability and performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-14
AI Technical Summary
The continuous increase in memory usage of log-based caching services has led to system crashes and sluggish business performance. Existing technologies are unable to effectively control cache resources, resulting in system crashes and business losses.
After IO data is written to the cache, it is written to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list. When the target data block is in a locked state, it is flushed to the persistence layer to clear the cache space. Combined with the token bucket mechanism to control IO data writing, the reasonable use of cache resources is ensured.
It effectively avoids disordered accumulation of cached data, prevents system crashes, ensures stable and efficient business operation, improves system performance and reliability, and avoids performance degradation caused by excessive cache resource consumption.
Smart Images
Figure CN121858290A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a cache management method, system, storage medium, and electronic device. Background Technology
[0002] In the EDS system, the log-based caching service acts as a caching layer, handling the aggregation of business I / O and the rational utilization of computing resources. It uses `vg` (the unit of management data for the log-based caching service) as the granularity to carry the operation logs of front-end business traffic and metadata services. Multiple worker instances are set up under each `vg` to improve concurrency with the persistence layer and achieve greater throughput. When each worker instance writes data, it uses `slog` to manage the data (organized in memory, with a log copy in the persistence layer). After a certain amount is aggregated, the I / O data is flushed to the persistence layer in units of `slog`.
[0003] The logging of metadata operations relies on deletions issued by the metadata service itself. However, when the log-based cache is flushed, the backend storage performance may be limited, leading to an accumulation of slog resources in the log-based cache service layer. This causes memory usage to rise continuously, potentially resulting in system crashes. Therefore, it is necessary to set an upper limit on the memory resources used by the log-based cache service.
[0004] To control resource limits, log-based caching services use slog to track resource usage. When each I / O operation reaches the cache, it checks if the resource limit has been reached. If the limit is reached, it immediately stops writing to the upper layer. However, this will cause severe performance jaggedness on the business side, and may even lead to business interruptions, which is unacceptable.
[0005] Therefore, how to achieve effective cache management is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] The purpose of this application is to provide a cache management method, system, storage medium, and electronic device that can manage cache resources and prevent system downtime or business disruption.
[0007] To address the aforementioned technical problems, this application provides a cache management method, the specific technical solution of which is as follows:
[0008] After the IO data is written to the cache, the IO data is written to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list;
[0009] After the IO data is written to the target management data block, it is determined whether the target management data block is in a locked state;
[0010] If so, first flush the IO data in the target management data block to the persistence layer, then delete the target management data block and clear the cache space corresponding to the target management data block.
[0011] Optionally, after IO data is written to the cache, the following also applies:
[0012] Determine whether the data block list contains any management data blocks that are not fully filled;
[0013] If so, execute the step of writing the IO data to the target management data block with the longest lifespan among the management data blocks that are not yet full in the data block list;
[0014] If the data block list is full of management data blocks, request a new management data block and add the new management data block to the data block list. Create a cache space corresponding to the new management data block in the cache as a memory copy of the new management data block.
[0015] Optionally, after flushing the IO data in the target management data block to the persistent layer, the method further includes:
[0016] Record the data block information in the management data block that has been flushed; the data size in the data block information is used to calculate the flushing bandwidth from the management data block to the persistence layer.
[0017] Optionally, before writing IO data to the cache, the following may also be included:
[0018] Request the data token corresponding to the IO data from the token bucket;
[0019] If the data token is obtained, the IO data is written to the cache.
[0020] Optionally, before requesting the data token corresponding to the IO data from the token bucket, the method further includes:
[0021] Generate the token bucket;
[0022] The token generation speed in the token bucket is set according to the downsweep bandwidth and the current storage level of the cache; wherein the downsweep bandwidth and the token generation speed are positively correlated, and the storage level is negatively correlated with the token generation speed.
[0023] Optional, also includes:
[0024] Receive the metadata corresponding to the IO data and write the metadata to the target management data block;
[0025] Accordingly, after determining that the target management data block is in a locked state, the metadata in the target management data block is flushed to the key-value database.
[0026] Optional, also includes:
[0027] When a process restart is detected, if the current storage level of the cache is at the lower storage limit, the metadata of the IO data corresponding to the process is loaded into memory.
[0028] This application also provides a cache management system, the specific technical solution of which is as follows:
[0029] The IO data caching module is used to write the IO data to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list after the IO data is written to the cache;
[0030] The management data block verification module is used to determine whether the target management data block is in a locked state after the IO data is written to the target management data block;
[0031] The cache management module is used to first flush the IO data in the target management data block to the persistence layer when the judgment result of the management data block verification module is yes, and then delete the target management data block and clear the cache space corresponding to the target management data block.
[0032] This application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the cache management method described above.
[0033] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the cache management method described above when it calls the computer program in the memory.
[0034] This application provides a cache management method, comprising: after writing IO data into the cache, writing the IO data into the target management data block with the longest lifespan among the unfilled management data blocks in the data block list; after the IO data is written into the target management data block, determining whether the target management data block is in a locked state; if so, first flushing the IO data in the target management data block to the persistence layer, then deleting the target management data block, and clearing the cache space corresponding to the target management data block.
[0035] This application achieves effective integration and orderly management of cached data by writing IO data to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list after the IO data is written to the cache, thus avoiding the disorderly accumulation of cached data. Subsequently, by determining whether the target management data block is in a locked state, if the second management data block is in a locked state, it is first flushed to the persistence layer to ensure the persistent storage of IO data and prevent IO data loss. Then, the target management data block is deleted and its corresponding cache space is cleared, promptly releasing cache resources that have been occupied for a long time. Therefore, this application, by combining a precise release mechanism for dynamic management of cache resources, can effectively control the usage of cache resources, maintaining them within a reasonable range and preventing excessive cache resource occupation that could lead to system resource shortages, thereby avoiding the risk of system downtime. At the same time, it can also avoid serious performance degradation caused by excessive cache resource occupation, ensuring stable and efficient operation of the business and improving the overall system performance and reliability.
[0036] This application also provides a cache management system, a computer-readable storage medium, and an electronic device, which have the above-mentioned beneficial effects, and will not be elaborated here. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0038] Figure 1 A flowchart illustrating a cache management method provided in an embodiment of this application;
[0039] Figure 2 A flowchart illustrating a cache management process provided in an embodiment of this application;
[0040] Figure 3 This is a schematic diagram of the token bucket rate limiting principle provided in an embodiment of this application;
[0041] Figure 4 This is a schematic diagram illustrating the relationship between token generation speed and storage level provided in an embodiment of this application;
[0042] Figure 5 This is a schematic diagram of a cache management system structure provided in an embodiment of this application;
[0043] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0045] See Figure 1 , Figure 1 A flowchart of a cache management method provided in this application embodiment, the method including:
[0046] S101: After the IO data is written to the cache, the IO data is written to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list;
[0047] S102: After the IO data is written to the target management data block, determine whether the target management data block is in a locked state; if so, proceed to step S103.
[0048] S103: First, flush the IO data in the target management data block to the persistence layer, then delete the target management data block and clear the cache space corresponding to the target management data block.
[0049] This embodiment targets business I / O data and performs cache management after I / O data is written to the cache. Specifically, after the I / O data is written to the cache, it is first written to the target management data block in the data block list. This target management data block is the management data block in the data block list that is not full and has the longest lifespan. It should be noted that for scenarios with a large number of concurrent I / O data writes, the process of determining the target management data block to be written for each I / O data is a serial process, while writing the I / O data to its corresponding target management data block after determining the target management data block can be a concurrent process.
[0050] In one feasible implementation, after writing IO data to the cache, it can be first determined whether there are any unfilled management data blocks in the data block list. If there are unfilled management data blocks in the data block list, the IO data is written to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list.
[0051] If the data block list consists entirely of full management data blocks, a new management data block can be requested and added to the data block list. The cache space corresponding to the new management data block is then created as a memory copy of the new management data block. This memory copy improves the read speed of I / O data, allowing for faster I / O data hits. Furthermore, when determining whether a target management data block is locked, the memory copy can quickly identify whether the target management data block is full.
[0052] This embodiment assumes that the data block list has been pre-built, and the data block list contains several management data blocks. The management data block is the unit of data management. Multiple IO data can exist on a management data block. Once a management data block is locked, no new IO data can be written.
[0053] In addition, the lifecycle of each managed data block in the data block list can be recorded. For example, a lifecycle attribute can be set to record information such as the creation time and last access time of the managed data block. Using this information, the lifecycle length of the managed data block can be calculated.
[0054] In step S102, the locking status of the target managed data block is monitored in real time within the data block list. After each IO data write, it is determined whether the target managed data block being written to is in a locked state. A locked state indicates that the managed data block is full; in this state, new IO data writing is prohibited. The locking status of each managed data block can be identified by setting a locking flag in the data block or using other synchronization mechanisms (such as mutexes, semaphores, etc.). When it is necessary to determine whether a managed data block is in a locked state, the locking flag or related synchronization status of that managed data block can be checked.
[0055] In step S103, the storage medium corresponding to the persistent layer is not specifically limited and can include, but is not limited to, disks or solid-state drives. Simultaneously, during the flushing process to the persistent layer, relevant parameters of the persistent layer, such as storage path, storage format, and data block size, need to be configured by default to ensure that IO data can be correctly stored in the persistent layer. Furthermore, a mapping relationship needs to be established between the persistent layer and the management data block so that the storage location of the management data block in the persistent layer can be accurately located during the flushing operation. The flushing operation refers to the process of transferring IO data from the management data block to the persistent layer. During the flushing process, the IO data in the target management data block must be completely written to the persistent layer to ensure persistent data storage. The flushing operation can be completed through the underlying storage interface and driver, and the status of the target management data block needs to be updated, marking it as flushed. Afterward, the second management data block can be deleted from the data block list, and the corresponding cache space of the second management data block can be cleared. Cache space can be cleared by zeroing out the data in the cache area or marking it as invalid, so that the cache space can be reused later.
[0056] See Figure 2 , Figure 2 This is a flowchart illustrating a cache management process provided in an embodiment of this application. Figure 2 In this system, a slog is used to identify managed data blocks, and the slog list represents the list of data blocks. After IO data is written, it can be determined whether a new slog needs to be requested. The flushing process is independent of the IO data writing to the slog. However, flushing of the target managed data block can only be performed after the IO data has been written to the target managed data block and the target managed data block is in a locked state.
[0057] In addition, the IO data written to the persistence layer can be synchronized to the cache and a success message can be returned to the upper layer to ensure that the IO data has been successfully written to the persistence layer, guaranteeing data consistency and integrity, and preventing the upper layer application from thinking that the IO data writing has failed and thus attempting to write it again.
[0058] This application embodiment achieves effective integration and orderly management of cached data by writing IO data into the target management data block with the longest lifespan among the unfilled management data blocks in the data block list after the IO data is written to the cache, thus avoiding the disorderly accumulation of cached data. Subsequently, by determining whether the target management data block is in a locked state, if the second management data block is in a locked state, it is first flushed to the persistence layer to ensure the persistent storage of IO data and prevent IO data loss. Then, the target management data block is deleted and its corresponding cache space is cleared, promptly releasing cache resources that have been occupied for a long time. It is evident that this application, by combining a precise release mechanism for dynamic management of cache resources, can effectively control the usage of cache resources, maintaining them within a reasonable range, preventing excessive cache resource occupation that could lead to system resource shortages, and thus avoiding the risk of system downtime. Simultaneously, it can also avoid serious performance degradation caused by excessive cache resource occupation, ensuring stable and efficient operation of services and improving the overall system performance and reliability.
[0059] Based on the above embodiments, as a preferred embodiment, in order to further control the cache space and prevent cache resources and memory resources from becoming uncontrolled, restrictions can be placed on IO data writes. The specific process can be as follows:
[0060] Before writing IO data to the cache, request a data token corresponding to the IO data from the token bucket. If the data token is obtained, write the IO data to the cache.
[0061] Before writing IO data to the cache, a data token from the token bucket must be obtained. This embodiment assumes the token bucket has been pre-generated; for example, it can be created during cache initialization. In one feasible implementation, the token generation rate in the token bucket can be set based on the brush bandwidth and the current cache storage level. Furthermore, the brush bandwidth and the token generation rate are positively correlated, while the storage level is negatively correlated with the token generation rate.
[0062] The refresh bandwidth can be monitored in real time or periodically. Refresh bandwidth refers to the bandwidth used to transmit cached data to downstream storage or other processing stages, and can be obtained through network interface monitoring tools, system performance indicator acquisition modules, etc. When the refresh bandwidth increases, cached data can be processed and transmitted more quickly. At this time, the token generation speed should be increased accordingly to allow more IO data to be written to the cache, fully utilizing bandwidth resources and avoiding data accumulation due to limited cache writes.
[0063] See Figure 3 , Figure 3 This is a schematic diagram of the token bucket rate limiting principle provided in an embodiment of this application. Figure 3As can be seen from this, the token generation speed of the token bucket is set based on the refresh speed, i.e., the release bandwidth and the cache water level. By requiring that the data tokens written to the IO data cache must be included in the token bucket, the writing of IO data to the cache is restricted.
[0064] In one feasible implementation, in order to improve the statistical accuracy of the down-brush bandwidth, the statistical interval of the down-brush bandwidth can be set to an integer multiple of the number of concurrent data blocks being managed during down-brush.
[0065] Simultaneously, the current cache storage level can be monitored. The storage level refers to the percentage of cache-occupied storage space to total storage space. The storage level can be obtained using the storage level query interface provided by the cache management system or by periodically calculating the cache data volume and total capacity. When the cache storage level rises, it indicates that cache space is gradually becoming strained. In this case, the token generation rate should be reduced, and the speed of IO data writing to the cache should be slowed down to prevent cache overflow and ensure the stability of the cache system.
[0066] See Figure 4 , Figure 4 This is a schematic diagram illustrating the relationship between token generation speed and cache level provided in an embodiment of this application. Figure 4 The linear relationship between token generation speed and cache level is illustrated by an example. Clearly, as the cache level increases, the token generation speed decreases, and the slope of the corresponding line segment also decreases, from 1.2 to 0.8. Those skilled in the art can also use other methods to dynamically adjust the token generation speed in the token bucket, and the relationship between cache level and token generation speed does not necessarily have to be linear; these are not all limited to examples here.
[0067] This embodiment dynamically adjusts the token generation speed in the token bucket by comprehensively considering the dynamic changes in brush bandwidth and cache storage level, so that the token bucket can adapt to different IO data pressures.
[0068] When IO data is generated or received and needs to be written to the cache, a token request process is triggered. The IO data write request is sent to the token bucket, requesting data tokens corresponding to the amount of data. The token bucket processes the requests based on the current number of tokens and the generation rate. If there are enough tokens in the token bucket, they are allocated to the IO data write request; if there are not enough tokens in the token bucket, the IO data write request enters a waiting state, waiting for a token to be generated or released.
[0069] Once an IO data write request successfully acquires a data token, the IO data can be written to the cache. The write process follows the cache's write rules and mechanisms, such as specifying the cache address and updating the cache data structure, to ensure that the data is correctly stored in the cache. After the IO data is written to the cache, the corresponding data token is released, reducing the number of tokens in the token bucket so that subsequent IO data write requests can acquire the correct number of tokens. Simultaneously, the token bucket continuously generates new tokens according to a set token generation rate to maintain its normal operation and provide continuous token support for subsequent IO data writes.
[0070] In addition, during the token application process before writing IO data to the cache, timeout mechanisms can be set for token application failures (such as excessively long waiting time, no tokens being generated in the token bucket for a long time), priority sorting or rollback processing of waiting IO data can be performed to avoid the impact of long waiting times for individual IO data write requests on the overall system performance and business processes.
[0071] This embodiment introduces a token bucket mechanism to correlate front-end service throughput with refresh bandwidth and cache level. By controlling the balance between front-end and refresh bandwidth, it prevents uncontrolled cache memory resources and ensures smooth front-end service operation, avoiding performance jaggedness.
[0072] Based on the above embodiments, as a preferred embodiment, corresponding cache management can also be performed on metadata, and the specific process is as follows:
[0073] Step 1: Receive the metadata corresponding to the IO data and write the metadata to the target management data block;
[0074] The second step is to determine that the target management data block is in a locked state, and then flush the metadata in the target management data block to the key-value database.
[0075] Provided that the target management data block capacity is sufficient, IO data and its corresponding metadata need to be written to the same management data block.
[0076] If IO data and its corresponding metadata are not written at the same time, resulting in insufficient space in the target management data block for storing IO data, a new management data block can be requested. After successfully requesting a new management data block, a corresponding cache space is created as a memory copy of the new management data block. If the target management data block corresponding to the log-type caching service is found to be locked, the metadata in the target management data block is flushed to the key-value database. IO data and its corresponding metadata are written to the management data block. In abnormal situations, such as process restarts, the corresponding cached data is lost. In this case, the IO data and metadata stored in the management data block can be used for quick data recovery.
[0077] As can be seen, for management data blocks that store both IO data and their corresponding metadata, the IO data is written to disk during the flush process (i.e., flushed to the persistence layer), and the metadata is flushed to the key-value database. Afterward, the management data block is removed from the data block list, and the corresponding cache space is released.
[0078] Based on the above embodiments, in one feasible implementation, when a process restart is detected, if the current storage level of the cache is at the lower storage limit, the metadata of the IO data corresponding to the process is loaded into memory. When a process restarts, to utilize the high performance of the cache, data needs to be reloaded into the cache. Upon recovery, if the current cache occupancy has reached its maximum value (i.e., the cache level has reached the lower storage limit), loading the IO data into the cache can be abandoned, and only the metadata of the IO data is loaded into memory. This prevents new data from replacing potentially more useful data in the cache, maintaining the validity and hit rate of the data in the cache, ensuring that the cache can better serve the data that truly needs fast access later, thereby improving overall performance. Simultaneously, loading only the metadata of the IO data into memory, compared to completely reloading a large amount of data into the cache, significantly reduces the amount of data loaded and the time, allowing the process to recover from the restart state to a normally functioning state more quickly, shortening service interruption time. Furthermore, compared to loading the complete data, it occupies less memory space, avoiding excessive consumption of memory resources due to loading large amounts of data, allowing memory resources to be more rationally allocated to other processes or tasks that need them, improving the overall resource utilization of the system.
[0079] See Figure 5 , Figure 5 This is a schematic diagram of a cache management system provided in an embodiment of this application. The system includes:
[0080] The IO data caching module is used to write the IO data to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list after the IO data is written to the cache;
[0081] The management data block verification module is used to determine whether the target management data block is in a locked state after the IO data is written to the target management data block;
[0082] The cache management module is used to first flush the IO data in the target management data block to the persistence layer when the judgment result of the management data block verification module is yes, and then delete the target management data block and clear the cache space corresponding to the target management data block.
[0083] Based on the above embodiments, as a preferred embodiment, it further includes:
[0084] The data block detection module is used to determine whether there are any unfilled management data blocks in the data block list; if so, it proceeds to the IO data cache module.
[0085] The data block request module is used to request a new management data block if the data block list is full of management data blocks, add the new management data block to the data block list, and create a cache space corresponding to the new management data block in the cache as a memory copy of the new management data block.
[0086] Based on the above embodiments, as a preferred embodiment, it further includes:
[0087] The data block information recording module is used to record the data block information in the management data block that has been flushed; the data size in the data block information is used to calculate the flushing bandwidth from the management data block to the persistence layer.
[0088] Based on the above embodiments, as a preferred embodiment, it further includes:
[0089] The token request module is used to request a data token corresponding to the IO data from the token bucket; if the data token is obtained, the IO data is written to the cache.
[0090] Based on the above embodiments, as a preferred embodiment, it further includes:
[0091] A token bucket generation module is used to generate the token bucket; and to set the token generation speed in the token bucket according to the brush bandwidth and the current storage level of the cache; wherein the brush bandwidth and the token generation speed are positively correlated, and the storage level is negatively correlated with the token generation speed.
[0092] Based on the above embodiments, as a preferred embodiment, it further includes:
[0093] The process restart processing module is used to load the metadata of the IO data corresponding to the process into memory when a process restart is detected, if the current storage level of the cache is at the lower limit value.
[0094] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, can implement the steps of the methods provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0095] This application also provides an electronic device that may include a memory and a processor. The memory stores a computer program, and when the processor invokes the computer program in the memory, it can implement the steps of the method provided in the above embodiments. Of course, the electronic device may also include various network interfaces, power supplies, and other components. Please see [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device in this embodiment may include: a processor 2101 and a memory 2102.
[0096] Optionally, the electronic device may also include a communication interface 2103, an input unit 2104, a display 2105, and a communication bus 2106.
[0097] The processor 2101, memory 2102, communication interface 2103, input unit 2104, and display 2105 all communicate with each other through the communication bus 2106.
[0098] In this embodiment of the application, the processor 2101 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, an off-the-shelf programmable gate array, or other programmable logic devices.
[0099] The processor can call programs stored in memory 2102. Specifically, the processor can execute the operations performed by the electronic device in the above embodiments.
[0100] The memory 2102 is used to store one or more programs, which may include program code, including computer operation instructions. In this embodiment, the memory stores at least a program for implementing the following functions:
[0101] After the IO data is written to the cache, the IO data is written to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list;
[0102] After the IO data is written to the target management data block, it is determined whether the target management data block is in a locked state;
[0103] If so, first flush the IO data in the target management data block to the persistence layer, then delete the target management data block and clear the cache space corresponding to the target management data block.
[0104] In one possible implementation, the memory 2102 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during the use of the computer.
[0105] In addition, memory 2102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.
[0106] The communication interface 2103 can be an interface for a communication module, such as the interface for a GSM module.
[0107] This application may also include a display 2105 and an input unit 2104, etc.
[0108] Figure 6 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than [other components]. Figure 6 More or fewer components as shown, or combinations of certain components.
[0109] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. As the system provided in the embodiments corresponds to the method provided in the embodiments, the description is relatively simple; relevant parts can be found in the method section.
[0110] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.
[0111] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A cache management method, characterized in that, include: After the IO data is written to the cache, the IO data is written to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list; After the IO data is written to the target management data block, it is determined whether the target management data block is in a locked state; If so, first flush the IO data in the target management data block to the persistence layer, then delete the target management data block and clear the cache space corresponding to the target management data block.
2. The cache management method according to claim 1, characterized in that, After IO data is written to the cache, the following is also included: Determine whether the data block list contains any management data blocks that are not fully filled; If so, execute the step of writing the IO data to the target management data block with the longest lifespan among the management data blocks that are not yet full in the data block list; If the data block list is full of management data blocks, request a new management data block and add the new management data block to the data block list. Create a cache space corresponding to the new management data block in the cache as a memory copy of the new management data block.
3. The cache management method according to claim 1, characterized in that, After flushing the IO data in the target management data block to the persistent layer, the process further includes: Record the data block information in the management data block that has been flushed; the data size in the data block information is used to calculate the flushing bandwidth from the management data block to the persistence layer.
4. The cache management method according to claim 3, characterized in that, Before writing IO data to the cache, the following is also included: Request the data token corresponding to the IO data from the token bucket; If the data token is obtained, the IO data is written to the cache.
5. The cache management method according to claim 4, characterized in that, Before requesting the data token corresponding to the IO data from the token bucket, the process also includes: Generate the token bucket; The token generation speed in the token bucket is set according to the downsweep bandwidth and the current storage level of the cache; wherein the downsweep bandwidth and the token generation speed are positively correlated, and the storage level is negatively correlated with the token generation speed.
6. The cache management method according to claim 1, characterized in that, Also includes: Receive the metadata corresponding to the IO data and write the metadata to the target management data block; Accordingly, after determining that the target management data block is in a locked state, the metadata in the target management data block is flushed to the key-value database.
7. The cache management method according to any one of claims 1 to 6, characterized in that, Also includes: When a process restart is detected, if the current storage level of the cache is at the lower storage limit, the metadata of the IO data corresponding to the process is loaded into memory.
8. A cache management system, characterized in that, include: The IO data caching module is used to write the IO data to the target management data block with the longest lifespan among the unfilled management data blocks in the data block list after the IO data is written to the cache; The management data block verification module is used to determine whether the target management data block is in a locked state after the IO data is written to the target management data block; The cache management module is used to first flush the IO data in the target management data block to the persistence layer when the judgment result of the management data block verification module is yes, and then delete the target management data block and clear the cache space corresponding to the target management data block.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the cache management method as described in any one of claims 1-7.
10. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program, and the processor, when calling the computer program in the memory, implements the steps of the cache management method as described in any one of claims 1-7.