Method, system, device and equipment for controlling page cache of file cache system

Abstract

The embodiment of the invention provides a page cache control method, system, device and equipment for a file cache system, and the method comprises the steps: obtaining the number of used memory pages of a page cache pool, then comparing the number of used memory pages with a preset number of memory pages, and when the number of used memory pages is greater than the preset number of memory pages, controlling the page cache of the file cache system. According to the method, the speed of acquiring system resources by a front end is reduced, and the speed of processing the system resources by a rear end is improved, so that the number of available memory pages of the page cache pool is adjusted, and in this way, the stability of file cache in coping with complex business scenes, especially large-throughput scenes, can be ensured; and the problems of violent fluctuation or zeroing and the like of the performance are effectively solved, so that the stability and the robustness of the cache system are improved.

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a method, system, device and equipment for controlling page cache of a file cache system. Background Art

[0002] In order to improve the efficiency of file reading and writing, the file storage system uses page cache technology, which caches data in memory to reduce direct access to the disk, thereby accelerating data reading and writing. Page cache is usually maintained in the form of a page resource pool, which can flexibly respond to complex file storage business needs, including random reading and writing, interleaved reading and writing, large concurrency scenarios, and high IOPS scenarios.

[0003] However, in high-throughput front-end business scenarios, page cache consumption is particularly significant. Since the page cache recovery process involves back-end data flushing operations and the scheduling of the recovery subsystem, this process easily leads to uneven resource usage, which in turn causes unstable page cache usage. This instability manifests itself as drastic fluctuations in front-end performance, such as performance returning to zero or obvious glitches, which seriously affects the overall system performance and user experience. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a method, system, apparatus and device for controlling page cache of a file cache system that overcomes the above problems or at least partially solves the above problems.

[0005] In order to solve the above problem, an embodiment of the present invention discloses a method for controlling page cache of a file cache system, the method comprising:

[0006] Get the number of used memory pages of the page cache pool;

[0007] Comparing the used memory page quantity with a preset memory page quantity;

[0008] When the number of used memory pages is greater than the preset number of memory pages, the speed at which the front end acquires system resources is reduced, and the speed at which the back end processes the system resources is increased, so as to adjust the number of available memory pages in the page cache pool.

[0009] Optionally, reducing the speed at which the front end acquires system resources and increasing the speed at which the back end processes the system resources includes:

[0010] reducing the speed at which the front end obtains the system resources by a speed limiter;

[0011] And, the speed at which the backend processes the system resources is increased by the speed limiter.

[0012] Optionally, the reducing the speed at which the front end acquires the system resources by using a speed limiter, and increasing the speed at which the back end processes the system resources by using the speed limiter, includes:

[0013] By transferring a function, based on the usage rate of the page cache pool, controlling the rate limiter to reduce the speed at which the front end obtains the system resources;

[0014] And, through the transfer function, based on the usage rate of the page cache pool, the speed limiter is controlled to reduce the speed at which the front end obtains the system resources.

[0015] Optionally, reducing the speed at which the front end acquires the system resource by a speed limiter includes:

[0016] The speed of processing the front-end request is reduced by the rate limiter.

[0017] Optionally, increasing the speed at which the backend processes the system resources by the speed limiter includes:

[0018] Increasing the disk flushing frequency of the backend by the speed limiter; disk flushing means persistently storing the dirty data pages in the page cache pool;

[0019] Also, increase the number of memory pages flushed each time.

[0020] Optionally, the rate limiter is implemented based on a token bucket.

[0021] Accordingly, an embodiment of the present invention discloses a file cache system, wherein the file cache system is connected to a front end and a back end;

[0022] The front end is used to send a request to the file cache system;

[0023] The backend is used to flush the page cache pool to disk, and flushing the disk means that dirty data pages in the page cache pool are persistently stored;

[0024] The file cache system is used to obtain the number of used memory pages of the page cache pool; compare the number of used memory pages with the preset number of memory pages; when the number of used memory pages is greater than the preset number of memory pages, reduce the speed at which the front end obtains system resources and increase the speed at which the back end processes the system resources to adjust the number of available memory pages in the page cache pool.

[0025] Optionally, the file cache system is used to reduce the speed at which the front end obtains the system resources through a speed limiter; and to increase the speed at which the back end processes the system resources through the speed limiter.

[0026] Optionally, the file cache system is used to control the speed limiter to reduce the speed at which the front end obtains the system resources based on the usage rate of the page cache pool through a transfer function; and to control the speed limiter to reduce the speed at which the front end obtains the system resources based on the usage rate of the page cache pool through the transfer function.

[0027] Optionally, the file cache system is used to reduce the speed of processing the front-end request through the speed limiter.

[0028] Optionally, the file cache system is used to increase the disk flushing frequency of the back end through the speed limiter; disk flushing means persistently storing dirty data pages in the page cache pool; and increasing the number of memory pages flushed each time.

[0029] Optionally, the rate limiter is implemented based on a token bucket.

[0030] Accordingly, an embodiment of the present invention discloses a control device for page cache of a file cache system, the device comprising:

[0031] The used memory page number acquisition module is used to obtain the number of used memory pages in the page cache pool;

[0032] A comparison module, used to compare the number of used memory pages with a preset number of memory pages;

[0033] The adjustment module is used to reduce the speed at which the front end obtains system resources and increase the speed at which the back end processes the system resources when the number of used memory pages is greater than the preset number of memory pages, so as to adjust the number of available memory pages in the page cache pool.

[0034] Optionally, the adjustment module includes:

[0035] An adjusting submodule, configured to reduce a speed at which the front end acquires the system resources through a speed limiter;

[0036] And, the rate limiter increases the speed at which the backend processes the system resources.

[0037] Optionally, the adjustment submodule includes:

[0038] an adjusting unit, configured to control the rate limiter to reduce a speed at which the front end obtains the system resources based on a usage rate of the page cache pool by transferring a function;

[0039] And, through the transfer function, based on the usage rate of the page cache pool, the speed limiter is controlled to reduce the speed at which the front end obtains the system resources.

[0040] Optionally, the adjustment submodule includes:

[0041] A reducing unit is used to reduce the speed of processing the front-end request through the speed limiter.

[0042] Optionally, the adjustment submodule includes:

[0043] An increasing unit, used to increase the disk flushing frequency of the back end through the speed limiter; disk flushing means that dirty data pages in the page cache pool are persistently stored;

[0044] Also, increase the number of memory pages flushed each time.

[0045] Optionally, the rate limiter is implemented based on a token bucket.

[0046] Correspondingly, an embodiment of the present invention discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various steps of the control method embodiment of the page cache of the above-mentioned file cache system are implemented.

[0047] Accordingly, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the control method embodiment of the page cache of the above-mentioned file cache system are implemented.

[0048] The embodiments of the present invention include the following advantages:

[0049] The control method of the page cache of the file cache system of the embodiment of the present invention obtains the number of used memory pages of the page cache pool, and then compares the number of used memory pages with the preset number of memory pages. When the number of used memory pages is greater than the preset number of memory pages, the speed at which the front end obtains system resources is reduced, and the speed at which the back end processes system resources is increased to adjust the number of available memory pages in the page cache pool. In this way, the stability of the file cache in dealing with complex business scenarios, especially high-throughput scenarios, can be ensured, and the problem of drastic performance fluctuations or zeroing is effectively improved, thereby improving the stability and robustness of the cache system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of the steps of a method for controlling page cache of a file cache system according to an embodiment of the present invention;

[0051] Figure 2 is a schematic diagram of a positive feedback mechanism of an embodiment of the present invention;

[0052] Figure 3 is a schematic diagram of a negative feedback mechanism of an embodiment of the present invention;

[0053] Figure 4 is a schematic diagram of the operation of a feedback control mechanism according to an embodiment of the present invention;

[0054] Figure 5 is a schematic diagram of a file cache system throughput comparison according to an embodiment of the present invention;

[0055] Figure 6 It is a structural block diagram of an embodiment of a control device for page cache of a file cache system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] In order to improve the efficiency of file reading and writing, the file storage system uses page cache technology, which caches data in memory to reduce direct access to the disk, thereby accelerating data reading and writing. Page cache is usually maintained in the form of a page resource pool, which can flexibly respond to complex file storage business needs, including random reading and writing, interleaved reading and writing, large concurrency scenarios, and high IOPS scenarios.

[0058] However, in high-throughput front-end business scenarios, page cache consumption is particularly significant. Since the page cache recovery process involves back-end data flushing operations and the scheduling of the recovery subsystem, this process easily leads to uneven resource usage, which in turn causes unstable page cache usage. This instability manifests itself as drastic fluctuations in front-end performance, such as performance returning to zero or obvious glitches, which seriously affects the overall system performance and user experience.

[0059] One of the core concepts of an embodiment of the present invention is to obtain the number of used memory pages of the page cache pool, and then compare the number of used memory pages with the preset number of memory pages. When the number of used memory pages is greater than the preset number of memory pages, the speed at which the front end obtains system resources is reduced, and the speed at which the back end processes system resources is increased to adjust the number of available memory pages in the page cache pool. In this way, the stability of the file cache in dealing with complex business scenarios, especially high-throughput scenarios, can be ensured, and the problem of drastic performance fluctuations or zeroing is effectively improved, thereby improving the stability and robustness of the cache system.

[0060] Reference Figure 1 , shows a flowchart of a method for controlling page cache of a file cache system according to an embodiment of the present invention, which may specifically include the following steps:

[0061] Step 101, obtaining the number of used memory pages of the page cache pool.

[0062] Specifically, the page cache pool is an important component used to manage memory in the operating system, especially in file system operations, where it is mainly used to cache data blocks in the file system to improve the performance of file reading and writing. The page cache pool is a portion of memory in the operating system kernel that is used to store data blocks read from the disk or to be written to the disk. These data blocks are usually pages in the file system, hence the name "page cache". When an application reads a file, the operating system first checks whether the data block of the file already exists in the page cache pool. If it does, it is read directly from the cache without accessing the disk, which greatly improves the reading speed. When an application writes a file, the data is usually written to the page cache pool first and then asynchronously written to the disk. This can reduce the frequency of disk I / O (Input / Output) operations and improve write performance. By caching frequently accessed data blocks, the page cache pool can reduce the number of accesses to the disk, thereby extending the disk life and improving the overall system performance. The working principle of the page cache pool is as follows: Read operation, when the application requests to read a file, the operating system will first check whether the data block of the file exists in the page cache pool. If it exists, the data is read directly from the cache. If not, the data is read from the disk and cached in the page cache pool; when the application writes a file, the data is usually written to the page cache pool first. The operating system can choose to write the data to the disk immediately or later. This delayed writing method is called "delayed write" or "write-back"; when the memory in the page cache pool is insufficient, the operating system needs to select some cache data blocks for replacement. The size of the page cache pool is usually adjusted dynamically, and the operating system allocates and releases memory based on the usage of system memory and the access pattern of the file system.

[0063] The number of used memory pages in the page cache pool refers to the number of memory pages that have been used in the current page cache pool. Get the number of used memory pages in the page cache pool.

[0064] Step 102, comparing the used memory page quantity and the preset memory page quantity.

[0065] Specifically, the preset number of memory pages can be set according to actual needs in actual applications. The embodiment of the present invention does not limit this. Comparing the number of used memory pages and the preset number of memory pages can better understand whether there is a problem of insufficient memory or memory waste in the page cache pool.

[0066] Step 103, when the number of used memory pages is greater than the preset number of memory pages, reduce the speed at which the front end acquires system resources and increase the speed at which the back end processes the system resources to adjust the number of available memory pages in the page cache pool.

[0067] Specifically, the number of available memory pages refers to the number of memory pages in the page cache pool that are currently not in use. These memory pages can be allocated to new processes or used for page cache. Knowing the number of available memory pages can help you monitor the system's memory usage and determine whether there is a risk of insufficient memory.

[0068] By comparing the number of used memory pages with the preset number of memory pages, when the number of used memory pages is greater than the preset number of memory pages, the file cache system may face memory pressure, and measures need to be taken to reduce memory usage to ensure stable operation of the system. The speed at which the front end obtains system resources can be reduced, which can reduce the consumption of system resources, thereby reducing the speed at which memory pages are used; the speed at which the back end processes the system resources can be increased, which can release more memory pages and reduce memory pressure, thereby effectively adjusting the number of available memory pages in the page cache pool and ensuring that the system maintains stable operation under memory pressure.

[0069] A negative feedback mechanism is added to the front-end business. The feedback information includes the decreasing speed of the number of available memory pages in the page cache pool and the difference between the number of used memory pages and the preset number of memory pages. The faster the number of available memory pages decreases, the faster the back-end recovery efficiency cannot meet the front-end business consumption, and the front-end business traffic is reduced through the speed limiter.

[0070] A positive feedback mechanism is added to the backend processing. The feedback information includes the growth rate of dirty data pages, the decrease rate of the number of available memory pages in the page cache pool, and the difference between the number of used memory pages and the preset number of memory pages. The faster the number of available memory pages in the page cache pool decreases, the faster the backend recovery efficiency cannot meet the frontend business consumption, and the backend processing speed needs to be improved.

[0071] The control method of the page cache of the file cache system of the embodiment of the present invention obtains the number of used memory pages of the page cache pool, and then compares the number of used memory pages with the preset number of memory pages. When the number of used memory pages is greater than the preset number of memory pages, the speed at which the front end obtains system resources is reduced, and the speed at which the back end processes system resources is increased to adjust the number of available memory pages in the page cache pool. In this way, the stability of the file cache in dealing with complex business scenarios, especially high-throughput scenarios, can be ensured, and the problem of drastic performance fluctuations or zeroing is effectively improved, thereby improving the stability and robustness of the cache system.

[0072] In the embodiment of the present invention, the step 103 of reducing the speed at which the front end acquires system resources and increasing the speed at which the back end processes the system resources may specifically include the following sub-steps:

[0073] Sub-step S11, reducing the speed at which the front end acquires the system resources by using a speed limiter.

[0074] Specifically, the speed at which the front-end obtains system resources is reduced through a rate limiter, which is a mechanism for controlling request rates, usually to prevent system overload or abuse.

[0075] Sub-step S12, increasing the speed at which the backend processes the system resources by using the speed limiter.

[0076] Specifically, the speed limiter is used to increase the speed of backend processing system resources to ensure that the system can still run stably under high load conditions.

[0077] In the embodiment of the present invention, reducing the speed at which the front end acquires the system resources by using a speed limiter, and increasing the speed at which the back end processes the system resources by using the speed limiter, includes:

[0078] By transferring a function, the speed limiter is controlled to reduce the speed at which the front end obtains the system resources based on the usage rate of the page cache pool.

[0079] And, through the transfer function, based on the usage rate of the page cache pool, the speed limiter is controlled to reduce the speed at which the front end obtains the system resources.

[0080] Specifically, by transferring functions, based on the usage rate of cache pool memory pages, the number of threads and other system capabilities, when the usage rate reaches different values, the speed limiter will be controlled to modify the speed at which the front end obtains system resources, and the speed at which the back end processes system resources will be controlled to speed up the release of resources.

[0081] In the embodiment of the present invention, sub-step S11 may specifically include the following steps:

[0082] The speed of processing the front-end request is reduced by the rate limiter.

[0083] Specifically, the speed of processing front-end requests is reduced through the speed limiter. For example, originally 100 requests can be processed in one second. After the speed of processing front-end requests is reduced, 50 requests can be processed in one second.

[0084] In the embodiment of the present invention, sub-step S12 may specifically include the following steps:

[0085] Increasing the disk flushing frequency of the backend by the speed limiter; disk flushing means persistently storing the dirty data pages in the page cache pool;

[0086] Also, increase the number of memory pages flushed each time.

[0087] Specifically, disk flushing refers to the process of writing data in memory to disk, which is usually used to persist data or ensure data reliability. This process ensures that data will not be lost in the event of power outages or other abnormal situations. Increasing the back-end disk flushing frequency through the speed limiter means increasing the frequency of writing memory data to disk and improving the efficiency of recycling cache pages in the page cache pool. Each disk flushing has a time interval, and the data in the corresponding number of memory pages is flushed to the underlying disk according to the corresponding parameters. There are two ways to increase the disk flushing speed: shortening the time interval can increase the disk flushing speed, that is, increasing the back-end disk flushing frequency and increasing the number of memory pages flushed each time. Since disk flushing consumes system resources, the disk flushing speed is controlled by feedback to the back-end to reduce system consumption.

[0088] In the embodiment of the present invention, the rate limiter is implemented based on a token bucket.

[0089] Specifically, the rate limiter based on token bucket is a common rate limit algorithm. It controls the request rate by maintaining a fixed-capacity token bucket. The token bucket algorithm can effectively control the request rate, support burst traffic, and is relatively simple to implement. The basic principle of the token bucket algorithm: token bucket, a token bucket is a bucket with a fixed capacity for storing tokens. A token is a credential that allows an operation to be performed; token generation, tokens are generated at a fixed rate (Rate) and placed in the bucket. For example, one token is generated per second; token consumption, when an operation needs to be performed, a token must be taken out of the bucket. If there are enough tokens in the bucket, the operation can be performed; otherwise, the operation is restricted or delayed; bucket capacity, the capacity of the bucket determines the maximum number of tokens that can be stored. If the bucket is full, the newly generated token will be discarded.

[0090] The implementation steps of the token bucket algorithm include: initialization, setting the capacity of the token bucket, setting the token generation rate, and initializing the number of tokens in the token bucket; token generation, generating tokens at a fixed rate and placing them in the bucket; if the bucket is full, the newly generated tokens will be discarded; operation request, when an operation needs to be performed, check whether there are enough tokens in the bucket; if there are enough tokens in the bucket, take a token from the bucket and allow the operation to be performed; if there are not enough tokens in the bucket, the operation is restricted or delayed until there are enough tokens in the bucket. Advantages of the token bucket algorithm: Smooth traffic, the token bucket algorithm can smooth traffic and avoid sudden traffic from impacting the system; the token bucket algorithm can adjust the token generation rate and the capacity of the bucket according to demand, and flexibly control the frequency of operations; the token bucket algorithm is simple to implement and suitable for various application scenarios.

[0091] There are many other ways to implement the rate limiter, such as the leaky bucket algorithm. The leaky bucket algorithm is similar to a bucket with a fixed capacity. Requests in the bucket flow out (leak out) at a fixed rate. If the bucket is full, new requests will be discarded. Implementation steps: Initialize the capacity and outflow rate of the bucket. When a request arrives, check whether the bucket is full. If not, put the request into the bucket. If it is full, discard the request. Processing requests in the bucket at a fixed rate can smooth the traffic and avoid the impact of sudden traffic on the system, but it cannot handle sudden traffic, which may cause request delays. For example, the fixed window counter, the basic principle is to divide the time into windows of fixed size (such as one window per second), count the number of requests in each window, and reject requests that exceed the limit. Implementation steps: Set the window size and request limit, reset the counter at the beginning of each window, and increase the counter for each request received. If the counter exceeds the limit, reject the request. Although the implementation is simple and suitable for simple flow control, burst traffic may occur at the window boundary, resulting in uneven traffic.

[0092] The main factors affecting the page cache pool: the page resource recovery process indirectly affects the front-end business. If the resource recovery speed is too slow, the available pages in the page cache pool will become fewer or even exhausted, and the front-end business will drop to zero; in the write process, resource recovery depends on the back-end disk flushing: when there is dirty data on the page, it is necessary to wait for the dirty data to be flushed before it can be recycled; the disk flushing process affects the front-end business: there is a critical area between the disk flushing process and the front-end read and write process, and too frequent disk flushing will affect the front-end performance.

[0093] The basic principle of automatic control theory is to measure and analyze the input and output of the system, use mathematical models to describe the dynamic behavior of the system, and design controllers to adjust the input of the system to achieve the desired output. Feedback control in automatic control theory is divided into positive feedback and negative feedback. Figure 2 A schematic diagram of a positive feedback mechanism of an embodiment of the present invention is shown, where R(S) is the resources required by the system application or the speed at which the system processes resources before the transfer function is updated; E(S) is the resources required by the system application or the speed at which the system processes resources after the result transfer function is updated by controlling the speed limiter based on the usage of the buffer pool. Positive feedback refers to the feedback information sent by the controlled part, which has a positive relationship with the control information, that is, the change of the system output will promote the change of the input signal. Figure 3 Schematic diagram of a negative feedback mechanism of an embodiment of the present invention. Negative feedback refers to feedback information sent by the controlled part, which has a negative relationship with the control information, that is, changes in the system output will inhibit changes in the input signal. Negative feedback compares the difference between the system output and the expected output, and adjusts the behavior of the system according to the difference, so that the system can work stably in the expected state. Negative feedback can improve the stability, accuracy and robustness of the system.

[0094] Reference Figure 4 , showing a schematic diagram of the operation of a feedback control mechanism according to an embodiment of the present invention, wherein R(s) is the input system, which refers to the initialization size of the page cache pool here, the symbol pointed to by the arrow of R indicates that the R(S) input is affected by the H(s) transfer function and needs to update the resource speed of the request, N(s) represents the interference system, which refers to the consumption of the page by the front-end business here, Figure 2 After the consumption of pages by the front-end business N(S) is processed by the negative feedback transfer function h1, N(S) changes from N1 to N2. G(s) represents the page cache pool. Figure 2 In the figure, the number of page cache pool resources G(S) is processed by the positive and negative feedback transfer functions. The state of G(S) changes from G1 to G2, which reduces the consumption speed (or increases the sub-resources of the resource pool after positive feedback). C(s) represents the output system, which refers to the adjusted available resource size of the page cache pool. H(s) represents the transfer function used to control the speed limiter, which is affected by the system's capabilities (such as scheduling threads, resource pool size) and input signals. The transfer function controls the speed limiter to modify the speed at which the front end obtains system resources, and controls the speed limiter to modify the speed at which the back end processes system resources, thereby accelerating the release of resources. One is to speed up the processing speed in the positive direction, and the other is to reduce the speed in the reverse direction. Both positive and negative feedback have H(S).

[0095] The transfer function H(S) is an important concept in control system theory, which is used to describe the input-output relationship of a linear time-invariant system (LTI system). The transfer function describes the response characteristics of the system to input signals of different frequencies. Through the transfer function, important characteristics such as the stability, frequency response, poles and zeros of the system can be analyzed. The transfer function can be constructed in the following ways: differential equations. For a linear time-invariant system, the transfer function can be solved by its differential equations. The differential equations are Laplace transformed to obtain the transfer function; system block diagrams. The transfer function of the system can be derived through the block diagram of the system. The transfer function of the entire system can be obtained by multiplying or adding the transfer functions of each subsystem; experimental data: The transfer function of the system can be fitted by experimentally measuring the input-output relationship of the system.

[0096] Reference Figure 5 , shows a schematic diagram of a file cache system throughput comparison of an embodiment of the present invention, using the kernel page cache as a prototype, implementing the page cache system to verify the present solution, the initial size of the page cache pool is 4096, using fio (Flexible I / O Tester) to write a 1GB file, the queue depth is 32, the IO size is 1M, and the sampling is 30s to verify the improvement of the present solution on the front-end business, refer to Figure 3, the horizontal axis represents time, and the vertical axis represents the throughput of the front end. Throughput refers to the amount of data or the number of requests that the system can process per unit time, usually expressed as the amount of data processed per second (such as bytes / second, megabytes / second) or the number of requests processed per second (such as requests / second, transactions / second). Before the correction system was added, the front-end business writes fluctuated greatly, and the entire system was in a state of fatigue. After the correction system was added, the front-end throughput was relatively stable; from the perspective of the overall system throughput, the total throughput within 30 seconds after the correction system was added was much greater than before the correction system was added.

[0097] The embodiment of the present invention adjusts the page cache in the file cache system through the feedback principle of automatic control, adds a negative feedback mechanism through the front-end business, adds a positive feedback mechanism and a transfer function regulator to the disk flushing system, introduces the feedback mechanism in the automatic control theory, and effectively realizes the intelligent management of resources to achieve the optimization of resource utilization and improve the stability and robustness of the page cache pool.

[0098] Feedback control based on automatic control theory is used to effectively implement intelligent management of resources, making the throughput of the system front end more stable and improving the stability and robustness of the cache system. The feedback control mechanism of control theory is added to dynamically adjust the cache resource allocation in different processes and provide automatically adaptive optimal solutions for resource scheduling of each process. It is applied to the cache scheduling of the file storage system, and intelligently adjusts the cache scheduling through feedback control to ensure the stability of the file cache in dealing with complex business scenarios, especially high-throughput scenarios.

[0099] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0100] In an embodiment of the present invention, a file cache system is disclosed, wherein the file cache system is connected to a front end and a back end;

[0101] The front end is used to send a request to the file cache system;

[0102] The backend is used to flush the page cache pool to disk, and flushing the disk means that dirty data pages in the page cache pool are persistently stored;

[0103] The file cache system is used to obtain the number of used memory pages of the page cache pool; compare the number of used memory pages with the preset number of memory pages; when the number of used memory pages is greater than the preset number of memory pages, reduce the speed at which the front end obtains system resources and increase the speed at which the back end processes the system resources to adjust the number of available memory pages in the page cache pool.

[0104] In an embodiment of the present invention, the file cache system is used to reduce the speed at which the front end obtains the system resources through a speed limiter; and to increase the speed at which the back end processes the system resources through the speed limiter.

[0105] In an embodiment of the present invention, the file cache system is used to control the speed limiter to reduce the speed at which the front end obtains the system resources based on the usage rate of the page cache pool through a transfer function; and, to control the speed limiter to reduce the speed at which the front end obtains the system resources based on the usage rate of the page cache pool through the transfer function.

[0106] In an embodiment of the present invention, the file cache system is used to reduce the speed of processing the front-end request through the speed limiter.

[0107] In an embodiment of the present invention, the file cache system is used to increase the disk flushing frequency of the back end through the speed limiter; disk flushing means persistently storing dirty data pages in the page cache pool; and increasing the number of memory pages flushed each time.

[0108] In the embodiment of the present invention, the rate limiter is implemented based on a token bucket.

[0109] Reference Figure 6 , shows a structural block diagram of a control device for page cache of a file cache system according to an embodiment of the present invention, which may specifically include the following modules:

[0110] The used memory page number acquisition module is used to obtain the number of used memory pages in the page cache pool;

[0111] A comparison module, used to compare the number of used memory pages with a preset number of memory pages;

[0112] The adjustment module is used to reduce the speed at which the front end obtains system resources and increase the speed at which the back end processes the system resources when the number of used memory pages is greater than the preset number of memory pages, so as to adjust the number of available memory pages in the page cache pool.

[0113] The control device of the page cache of the file cache system of the embodiment of the present invention obtains the number of used memory pages of the page cache pool, and then compares the number of used memory pages with the preset number of memory pages. When the number of used memory pages is greater than the preset number of memory pages, the speed of the front-end acquiring system resources is reduced, and the speed of the back-end processing system resources is increased to adjust the number of available memory pages in the page cache pool. In this way, the stability of the file cache in dealing with complex business scenarios, especially high-throughput scenarios, can be ensured, and the problem of drastic performance fluctuations or zeroing is effectively improved, thereby improving the stability and robustness of the cache system.

[0114] In an embodiment of the present invention, the adjustment module includes:

[0115] An adjusting submodule, configured to reduce a speed at which the front end acquires the system resources through a speed limiter;

[0116] And, the rate limiter increases the speed at which the backend processes the system resources.

[0117] In an embodiment of the present invention, the adjustment submodule includes:

[0118] an adjusting unit, configured to control the rate limiter to reduce a speed at which the front end obtains the system resources based on a usage rate of the page cache pool by transferring a function;

[0119] And, through the transfer function, based on the usage rate of the page cache pool, the speed limiter is controlled to reduce the speed at which the front end obtains the system resources.

[0120] In an embodiment of the present invention, the adjustment submodule includes:

[0121] A reducing unit is used to reduce the speed of processing the front-end request through the speed limiter.

[0122] In an embodiment of the present invention, the adjustment submodule includes:

[0123] An increasing unit, used to increase the disk flushing frequency of the back end through the speed limiter; disk flushing means that dirty data pages in the page cache pool are persistently stored;

[0124] Also, increase the number of memory pages flushed each time.

[0125] In the embodiment of the present invention, the rate limiter is implemented based on a token bucket.

[0126] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0127] An embodiment of the present invention further provides an electronic device, including:

[0128] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the control method embodiment of the page cache of the above-mentioned file cache system are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0129] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the control method embodiment of the page cache of the above-mentioned file cache system are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0130] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0131] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0132] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0136] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0137] The above is a detailed introduction to the control method, system, device and equipment of the page cache of a file cache system provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for controlling page cache of a file cache system, characterized in that: The method comprises: Get the number of used memory pages of the page cache pool; Comparing the used memory page quantity with a preset memory page quantity; When the number of used memory pages is greater than the preset number of memory pages, the speed at which the front end acquires system resources is reduced, and the speed at which the back end processes the system resources is increased, so as to adjust the number of available memory pages in the page cache pool.

2. The method according to claim 1, characterized in that The reducing the speed at which the front end acquires system resources and increasing the speed at which the back end processes the system resources comprises: reducing the speed at which the front end obtains the system resources by a speed limiter; And, the speed at which the backend processes the system resources is increased by the speed limiter.

3. The method according to claim 2, characterized in that The step of reducing the speed at which the front end acquires the system resources by using a speed limiter, and increasing the speed at which the back end processes the system resources by using the speed limiter, comprises: By transferring a function, based on the usage rate of the page cache pool, controlling the rate limiter to reduce the speed at which the front end obtains the system resources; And, through the transfer function, based on the usage rate of the page cache pool, the speed limiter is controlled to reduce the speed at which the front end obtains the system resources.

4. The method according to claim 2, characterized in that: The reducing the speed at which the front end obtains the system resource by using a speed limiter includes: The speed of processing the front-end request is reduced by the rate limiter.

5. The method according to claim 2, characterized in that: The increasing the speed at which the backend processes the system resources by the speed limiter includes: Increasing the disk flushing frequency of the backend by the speed limiter; disk flushing means persistently storing the dirty data pages in the page cache pool; Also, increase the number of memory pages flushed each time.

6. The method according to claim 3, characterized in that The rate limiter is implemented based on a token bucket.

7. A file cache system, characterized in that: The file cache system is connected to the front end and the back end; The front end is used to send a request to the file cache system; The backend is used to flush the page cache pool to disk, and flushing the disk means that dirty data pages in the page cache pool are persistently stored; The file cache system is used to obtain the number of used memory pages of the page cache pool; compare the number of used memory pages with the preset number of memory pages; when the number of used memory pages is greater than the preset number of memory pages, reduce the speed at which the front end obtains system resources and increase the speed at which the back end processes the system resources to adjust the number of available memory pages in the page cache pool.

8. A control device for page cache of a file cache system, characterized in that: The device comprises: The used memory page number acquisition module is used to obtain the number of used memory pages in the page cache pool; A comparison module, used to compare the number of used memory pages with a preset number of memory pages; The adjustment module is used to reduce the speed at which the front end obtains system resources and increase the speed at which the back end processes the system resources when the number of used memory pages is greater than the preset number of memory pages, so as to adjust the number of available memory pages in the page cache pool.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method for controlling the page cache of the file cache system according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for controlling the page cache of the file cache system according to any one of claims 1 to 5 are implemented.

Images