Cache management method and device for metadata management component

Abstract

The present application provides a cache management method and device for a metadata management component. The method includes: in response to the use of the metadata management component monitored in the current monitoring cycle being greater than a first cache threshold and less than a second cache threshold, calculating a cache change difference according to a preset time period corresponding to the current time and the change in the use of the metadata management component in the previous preset time period; determining the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; determining the number of sub-cache elements to be eliminated for each client based on the determined total number of cache elements and the weight and reference number of eliminated elements configured for each client; and performing cache elimination according to the determined number of sub-cache elements to be eliminated for each client. The method can improve the management capabilities of the metadata management component and improve customer experience.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a cache management method and device for a metadata management component. Background Art

[0002] At present, the cache eviction strategy in the cache management of the metadata management component (MDS) of the distributed file system service (CephFS) provided by Ceph is mainly a static cache eviction strategy; this static cache eviction strategy is to directly evict the cache according to the number of elements that can be cleared each time configured for the client when the used cache of MDS exceeds the configured maximum cache threshold.

[0003] This static cache eviction strategy has relatively simple eviction rules, and treats all clients equally according to pre-configured indiscriminate configurations. This is not flexible enough, reduces the management capabilities of the MDS, and provides a poor customer experience. Summary of the invention

[0004] In view of this, the present application provides a cache management method and device for a metadata management component, which can improve the management capability of the metadata management component and improve the customer experience.

[0005] To solve the above technical problems, the technical solution of this application is implemented as follows:

[0006] In one embodiment, a cache management method of a metadata management component is provided, the method comprising:

[0007] Periodically monitor the usage cache of metadata management components;

[0008] In response to the use cache of the metadata management component monitored in the current monitoring period being greater than a first cache threshold and less than a second cache threshold, a cache change difference is calculated according to a change amount of the use cache of the metadata management component in a preset time period corresponding to the current time, and a change amount of the use cache of the metadata management component in a previous preset time period; wherein the first cache threshold is less than the second cache threshold;

[0009] Determine the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated;

[0010] Determine the number of sub-cache elements to be removed from each client according to the total number of cache elements to be removed in the current monitoring cycle, as well as the weight configured for each client and the reference number of removed elements;

[0011] Perform cache culling according to the determined number of sub-cache elements to be culled by each client;

[0012] The determining of the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated includes:

[0013] Based on the comparison result of the cache change difference calculated in the current monitoring cycle and 0, determine that the total number of cache elements to be eliminated in the current monitoring cycle is N times the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; wherein N is 0, 1 or greater than 1;

[0014] The determining the number of sub-cache elements to be removed from each client according to the determined total number of cache elements to be removed in the current monitoring cycle, and the weight configured for each client and the reference number of removed elements, includes:

[0015] Determining the estimated number of sub-cache elements to be removed for each client according to the determined total number of cache elements to be removed in the current monitoring cycle and the weight configured for each client;

[0016] The estimated number of sub-cache elements to be removed corresponding to each of the clients is compared with the reference number of removed elements configured for each of the clients, and the larger number of cache elements is determined as the actual number of sub-cache elements to be removed corresponding to the client.

[0017] Wherein, in response to the use of cache of the metadata management component monitored in the current monitoring period being greater than the first cache threshold and less than the second cache threshold, before calculating the cache change difference corresponding to the current monitoring period according to the change of the use of cache of the metadata management component in the preset time period corresponding to the current time and the change of the use of cache of the metadata management component in the previous preset time period, the method further includes:

[0018] Determine whether there are caches for two consecutive preset time periods; if so, execute the step of calculating the cache change difference corresponding to the current monitoring cycle based on the change in the use of the cache of the metadata management component within the preset time period corresponding to the current time, and the change in the use of the cache of the metadata management component within the previous preset time period; otherwise, determine that the reference number of culling elements configured for each client is the number of sub-cache elements to be culled by the client, and perform cache culling according to the determined number of sub-cache elements to be culled by each client.

[0019] Wherein, the method further comprises:

[0020] Calculating the change in the usage cache of the metadata management component within the preset time period includes:

[0021] Three sub-time intervals are set for the preset time period, namely a first sub-time interval, a second sub-time interval and a third sub-time interval; wherein the first sub-time interval covers the second sub-time interval, and the second sub-time interval covers the third sub-time interval;

[0022] Calculate the cache change per second in the three sub-time periods respectively;

[0023] The weighted sum of the cache changes per second in the three sub-time periods is used to obtain the change in the cache usage of the metadata management component within the preset time period;

[0024] The calculating the cache change difference according to the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time and the change amount of the used cache of the metadata management component in the previous preset time period includes:

[0025] Calculate the change in the usage cache of the metadata management component within a preset time period corresponding to the current time;

[0026] Calculate the change in the usage cache of the metadata management component within the previous preset time period;

[0027] The difference between the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time and the change amount of the used cache of the metadata management component in the previous preset time period is calculated as the cache change difference corresponding to the current monitoring period.

[0028] Wherein, the comparison result of the cache change difference calculated based on the current monitoring period and 0 determines that the total number of cache elements to be eliminated in the current monitoring period is N times the total number of cache elements to be eliminated determined in the most recent monitoring period in which the cache was eliminated; wherein N is 0, 1 or greater than 1, including:

[0029] Compare the cache change difference with the size of 0, and determine whether the cache is growing;

[0030] If the cache change difference is greater than 0, then N is greater than 1;

[0031] If the cache change difference is equal to 0, and the cache usage is in a growth state, then N is greater than 1;

[0032] If the cache change difference is equal to 0, and the cache usage is not in the growth state, N is 0;

[0033] If the cache change difference is less than 0, N is 1.

[0034] Wherein, the method further comprises:

[0035] When the usage cache of the metadata management component monitored in the current monitoring period is not less than the second cache threshold, if a session request sent by a client is received and the execution logic configured for the client is failed, a message that the operation failed due to cache excess is replied to the client that sent the session request.

[0036] Wherein, the method further comprises:

[0037] When the usage cache of the metadata management component monitored in the current monitoring cycle is not less than the second cache threshold, if a session request sent by a client is received and the execution logic configured for the client is to wait, the client is blocked and waits for a preset time;

[0038] If the preset time is reached and the current cache usage of the metadata management component is not less than the second cache threshold, a message is replied to the client indicating that the operation failed due to a waiting timeout;

[0039] If the used cache of the metadata management component is lower than the second cache threshold within a preset time, the session request sent by the client is processed and a processing result is responded.

[0040] Wherein, the method further comprises:

[0041] When the usage cache of the metadata management component monitored in the current monitoring cycle is not less than the second cache threshold, if a session request sent by the client is received and the execution logic configured for the client is passed, the cache is skipped and the session request sent by the client is processed through the back-end file, and the processing result is responded.

[0042] In another embodiment, a cache management device of a metadata management component is provided, the device comprising:

[0043] A monitoring unit, used for periodically monitoring the usage cache of the metadata management component;

[0044] a calculation unit, configured to calculate a cache change difference according to a change amount of the used cache of the metadata management component in a preset time period corresponding to the current time and a change amount of the used cache of the metadata management component in a previous preset time period in response to the used cache of the metadata management component monitored in the current monitoring period being greater than a first cache threshold and less than a second cache threshold; wherein the first cache threshold is less than the second cache threshold;

[0045] A determination unit, configured to determine the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; and determine the number of sub-cache elements to be eliminated for each client based on the total number of cache elements to be eliminated in the current monitoring cycle determined, and the weight configured for each client and the reference number of eliminated elements;

[0046] A processing unit, configured to perform cache culling according to the determined number of sub-cache elements to be culled by each client;

[0047] Among them, the determination unit is specifically used to determine, based on the comparison result of the cache change difference calculated in the current monitoring cycle and 0, whether the total number of cache elements to be eliminated in the current monitoring cycle is N times the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; wherein N is 0, 1 or a value greater than 1; determine the estimated number of sub-cache elements to be eliminated for each client based on the determined total number of cache elements to be eliminated in the current monitoring cycle and the weight configured for each client; compare the estimated number of sub-cache elements to be eliminated corresponding to each client with the reference number of elimination elements configured for each client, and determine the larger value of the number of cache elements as the actual number of sub-cache elements to be eliminated corresponding to the client.

[0048] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a cache management method of a metadata management component when executing the program.

[0049] In another embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, a cache management method of a metadata management component is implemented.

[0050] It can be seen from the above technical scheme that in the above embodiment, when it is determined that cache culling is required by periodically monitoring the usage cache of the MDS, the total number of cache elements to be culled in the current monitoring cycle is determined based on the change in the usage cache in two consecutive preset time periods and the total number of cache elements to be culled determined in the most recent monitoring cycle in which the cache is culled; then the number of sub-cache elements to be culled for each client is determined based on the determined total number of cache elements and the weights and reference culling element numbers configured for each client, and cache culling is performed according to the determined number of sub-cache elements to be culled for each client. This scheme takes into account the cache change in two consecutive preset time periods, the total number of cache elements to be culled in the most recent monitoring cycle in which the cache is culled, and the weights and reference culling element numbers configured for each client when determining the number of sub-cache elements to be culled for each client in the current monitoring cycle, and can perform cache culling dynamically, accurately, and in combination with the situation of each client. Therefore, this scheme can improve the management capabilities of the metadata management component and improve customer experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of configurations performed for 6 different clients in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a cache management process of a metadata management component in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of a cache management process of another metadata management component in an embodiment of the present application;

[0055] Figure 4 This is a schematic diagram of three sub-time periods within a preset time period in an embodiment of the present application;

[0056] Figure 5 This is a schematic diagram of the process of calculating the cache change difference in an embodiment of the present application;

[0057] Figure 6 This is a flow chart of determining the total number of cache elements to be removed in the current monitoring cycle in an embodiment of the present application;

[0058] Figure 7 This is a schematic diagram of a scenario using a burst growth of cache in an embodiment of the present application;

[0059] Figure 8 This is a schematic diagram of a scenario in which a cache is steadily increased in an embodiment of the present application;

[0060] Fig. 9 This is a schematic diagram of a scenario in which a cache is used to be flat and unchanged in an embodiment of the present application;

[0061] Fig.10 This is a schematic diagram of a scenario in which cache growth is slowed down in an embodiment of the present application;

[0062] Fig.11 A schematic diagram of a process for determining the number of sub-cache elements to be removed by each client in an embodiment of the present application;

[0063] Fig.12 This is a schematic diagram of a processing flow when the usage cache of the MDS is not less than the second cache threshold in an embodiment of the present application;

[0064] Fig.13 This is a schematic diagram of the structure of a cache management device of a metadata management component in an embodiment of the present application;

[0065] Fig.14 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe the order or precedence of the objects. It should be understood that the data used in this way can be interchanged where appropriate, so that the implementation of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0068] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0069] At present, the cache eviction strategy in the cache management of the metadata management component (MDS) of the distributed file system service (CephFS) provided by Ceph is mainly a static cache eviction strategy; for this static cache eviction strategy, the maximum cache threshold and the number of elements to be evict for each client are configured, and the number of elements to be evict for each client is the same. Based on this static cache eviction strategy, when the MDS's used cache exceeds the configured maximum cache threshold, cache eviction is directly performed according to the number of elements to be evict configured for the client. For example, if there are currently five clients and the number of elements configured is 100, then this element eviction will evict a total of 500 cache elements.

[0070] This static cache eviction strategy has a relatively simple eviction rule and lacks flexibility. Since the current static cache eviction configuration is configured in advance and treats all clients equally, this eviction scheme that does not distinguish between customers will reduce the customer experience of some customers. In addition, when determining the number of elements to be evict from the cache, it is based only on the number of configured static elements, without considering the long-term changes in cache usage. If the number of elements to be evict from the cache is too large, the customer experience will be further reduced. If the number of elements to be evict from the cache is not enough, the cache usage cannot be controlled in time, reducing the management capabilities of the metadata management component.

[0071] Based on the above problems, a cache management method of a metadata management component is provided in an embodiment of the present application. By periodically monitoring the usage cache of the MDS, when it is determined that cache culling is required, the total number of cache elements to be culled in the current monitoring cycle is determined according to the change in the usage cache in two consecutive preset time periods, and the total number of cache elements to be culled determined in the most recent monitoring cycle in which the cache is culled; then the number of sub-cache elements to be culled for each client is determined according to the determined total number of cache elements, and the weights and reference culling element numbers configured for each client, and cache culling is performed according to the determined number of sub-cache elements to be culled for each client. This scheme takes into account the cache change in two consecutive preset time periods, the total number of cache elements to be culled in the most recent monitoring cycle in which the cache is culled, and the weights and reference culling element numbers configured for each client when determining the number of sub-cache elements to be culled for each client in the current monitoring cycle, and can perform cache culling dynamically, accurately, and in combination with the situation of each client. Therefore, this scheme can improve the management capabilities of the MDS and improve customer experience.

[0072] In the embodiment of the present application, before the cache management of the metadata management component, the following relevant configurations need to be performed:

[0073] A monitoring cycle is used to monitor the time interval of the MDS's cache usage. At each monitoring cycle, it is determined whether cache eviction needs to be performed based on the monitored MDS's cache usage.

[0074] The first cache threshold is a cache threshold used to execute cache culling. When the cache usage exceeds the first cache threshold, the dynamic cache culling strategy provided in the embodiment of the present application is executed, which is described in detail below; in order to prevent the cache usage from reaching the second threshold; for example, the first cache threshold can be set to 10GiB, and the second cache threshold can be set to 14GiB; there is no limitation on this, and it can be adaptively set according to the specific version of MDS and the usage environment.

[0075] The second cache threshold is the maximum cache value that can be used by the configured metadata management component, and the first cache threshold is less than the second cache threshold; before the cache usage reaches the second cache threshold, a dynamic cache eviction strategy has been attempted, but the growth of cache usage has not been controlled.

[0076] When the MDS usage cache is not less than the second cache threshold, the relevant processing logic must be executed. Therefore, the strategy configured for this situation is: configure different execution logics according to the different needs of different clients, such as configuring Pass for clients with higher weights, configuring Fail for users with lower weights, and configuring Wait for clients with intermediate weights. This is just an example. You can configure the execution logic for each client according to your needs. The three configurable execution logics are as follows:

[0077] Fail: When the used cache of the MDS is not less than the second cache threshold, the MDS replies to the client that sent the session request with a message that the operation failed due to cache limit exceeded.

[0078] Wait, the execution logic is that MDS lets the client sending the request block for a preset time until the cache usage is removed below the second preset threshold. If the cache usage is still not reduced to below the second preset threshold after the timeout, the client is replied with a message that the operation failed due to the waiting timeout; if the cache usage has been reduced to below the second preset threshold within this preset time, the session request sent by the client can be directly processed.

[0079] The preset waiting time here can be set according to the actual application scenario, such as 60 seconds, which is only an example and is not limited to this.

[0080] Pass: When the MDS cache usage is not less than the second preset threshold, the cache is skipped and the backend file is processed when processing the corresponding request. This may reduce performance, but the corresponding client request will still be processed. This configuration is usually set for clients with higher weights.

[0081] The above three execution logics are configured for different clients respectively. When the used cache of the MDS is not less than the second preset threshold and the client sends a session request, the logic is executed to implement a fine-grained cache overlimit behavior strategy.

[0082] The preset time period can be set to 60 minutes, but is not limited to this and can be configured according to actual applications. This configuration is used to count cache usage data within a period of time, to calculate the trend of cache changes within multiple intervals, and to calculate the cache change difference; the length of this preset time period is greater than the length of the monitoring cycle.

[0083] Configure a weight for each client: This weight will determine the number of elements to be removed for the client when the dynamic cache removal strategy is subsequently executed; the optional value can be 1-10, and the default value is 5. For different clients, weights can be assigned to the client based on the importance of the client's session and the client's session frequency. In the embodiment of this application, the larger the weight value, the smaller the number of elements to be removed. This ensures the access performance of clients with large weight values;

[0084] Reference to the number of elements to be removed: The number of elements to be removed for each client. This configuration is the basic number of elements to be removed for each client each time when the MDS uses cache that exceeds the first cache threshold. This configuration can be adjusted manually.

[0085] See also Figure 1 , Figure 1 This is a schematic diagram of the configuration for 6 different clients in an embodiment of the present application. Figure 1 Only different configurations for different clients are given, such as the weight of each client, the number of reference elements to be eliminated, and the logical strategy to be executed when the MDS's used cache is not less than the second cache threshold. The first cache threshold, the second cache threshold, and the preset time period are the same for all clients, that is, they are not differentiated, and can be configured for each client one by one. When adding a new client, you only need to configure these different configurations.

[0086] When a new client accesses, a weight is assigned to the client, the number of reference culled elements is configured, and the logic policy executed when the used cache of the MDS is not less than the second cache threshold.

[0087] In the specific implementation, a client can be identified by a Session, and then when performing relevant configuration for the client, it can also be implemented through the configuration of the Session.

[0088] The following is a detailed description of the cache management process of the metadata management component implemented in the embodiment of the present application in conjunction with the accompanying drawings.

[0089] See also Figure 2 , Figure 2 This is a schematic diagram of a cache management process of a metadata management component in an embodiment of the present application. The specific steps are:

[0090] Step 201: periodically monitor the usage cache of the metadata management component.

[0091] Step 202, in response to the fact that the used cache of the metadata management component monitored in the current monitoring period is greater than the first cache threshold and less than the second cache threshold, the cache change difference is calculated based on the change in the used cache of the metadata management component in the preset time period corresponding to the current time and the change in the used cache of the metadata management component in the previous preset time period.

[0092] The current time is the moment when the usage cache of the metadata management component is greater than the first cache threshold and less than the second cache threshold during the current monitoring period;

[0093] The preset time period corresponding to the current time, that is, the end time of the preset time period is the current time.

[0094] The first cache threshold is a configured cache threshold for performing cache culling, and the second cache threshold is a maximum cache value that can be used by the configured metadata management component; the first cache threshold is smaller than the second cache threshold.

[0095] Step 203 , determining the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated.

[0096] The most recent monitoring cycle in which the cache is eliminated is the monitoring cycle before the current monitoring cycle, and the monitoring cycle in which the cache is also eliminated may be the monitoring cycle before the current monitoring cycle, or may be one of the monitoring cycles before the current monitoring cycle.

[0097] The specific implementation of this step includes: based on the comparison result of the cache change difference calculated in the current monitoring cycle and 0, determining that the total number of cache elements to be eliminated in the current monitoring cycle is N times the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; wherein N is 0, 1 or greater than 1.

[0098] Step 204 , determining the number of sub-cache elements to be removed from each client according to the determined total number of cache elements to be removed in the current monitoring cycle, and the weight configured for each client and the reference number of removed elements.

[0099] Step 205: Cache culling is performed according to the determined number of sub-cache elements to be culled from each client.

[0100] In this embodiment, by periodically monitoring the usage cache of the MDS, when it is determined that cache culling is required, the total number of cache elements to be culled in the current monitoring cycle is determined based on the change in the usage cache within two consecutive preset time periods and the total number of cache elements to be culled determined in the most recent monitoring cycle in which the cache was culled; then the number of sub-cache elements to be culled for each client is determined based on the determined total number of cache elements and the weights and reference culling element numbers configured for each client, and cache culling is performed according to the determined number of sub-cache elements to be culled for each client. This solution takes into account the cache change in two consecutive preset time periods, the total number of cache elements to be culled in the most recent monitoring cycle in which the cache was culled, and the weights and reference culling element numbers configured for each client when determining the number of sub-cache elements to be culled for each client in the current monitoring cycle, and can perform cache culling dynamically, accurately, and in combination with the situation of each client. Therefore, this solution can improve the management capabilities of the MDS and improve customer experience.

[0101] See also Figure 3 , Figure 3 This is a schematic diagram of a cache management process of another metadata management component in an embodiment of the present application. The specific steps are:

[0102] Step 301: periodically monitor the usage cache of the metadata management component.

[0103] Step 302, in response to the fact that the used cache of the metadata management component monitored in the current monitoring period is greater than the first cache threshold and less than the second cache threshold, determine whether the cache has been used for two consecutive preset time periods and whether the current monitoring period is the first monitoring period to remove the cache.

[0104] The first cache threshold is a configured cache threshold for performing cache culling, and the second cache threshold is a maximum cache value that can be used by the configured metadata management component.

[0105] When the cache of two consecutive preset time periods is not used, or when the current monitoring cycle is the first monitoring cycle to remove the cache, the total number of cache elements to be removed in the current monitoring cycle used in the dynamic cache removal strategy in this embodiment cannot be calculated. Therefore, when these two conditions are met at the same time, or when any one of the conditions is met, the configured reference number of removal elements is used to perform cache removal, that is, execute step 303; when neither of the above two conditions is met, the dynamic cache removal strategy in the embodiment of the present application is used to determine the number of sub-cache elements to be removed, that is, execute step 304.

[0106] Step 303, if the cache of two consecutive preset time periods is not used, and / or the current monitoring cycle is the first monitoring cycle to remove the cache, then determine the reference removal element number configured for each client as the number of sub-cache elements to be removed by the client; execute 307.

[0107] In step 304, if the cache has been used for two consecutive preset time periods and the current monitoring cycle is not the first monitoring cycle in which the cache is to be removed, the cache change difference is calculated based on the change in the cache used by the metadata management component in the preset time period corresponding to the current time and the change in the cache used by the metadata management component in the previous preset time period.

[0108] Step 305 , determining the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated.

[0109] If the most recent monitoring period in which the cache is culled is the first monitoring period in which the cache is culled, the total number of cache elements to be culled determined by the most recent monitoring period in which the cache is culled is the sum of the number of reference culled elements configured for each client;

[0110] If the most recent monitoring cycle in which the cache is eliminated is not the first monitoring cycle in which the cache is eliminated, the total number of cache elements in the current monitoring cycle is calculated based on the total number of cache elements in the most recent monitoring cycle in which the cache is eliminated.

[0111] The specific implementation of this step includes: based on the comparison result of the cache change difference calculated in the current monitoring cycle and 0, determining that the total number of cache elements to be eliminated in the current monitoring cycle is N times the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; wherein N is 0, 1 or greater than 1.

[0112] Step 306 , determining the number of sub-cache elements to be removed from each client according to the determined total number of cache elements to be removed in the current monitoring cycle, and the weight configured for each client and the reference number of removed elements.

[0113] Step 307: Cache culling is performed according to the determined number of sub-cache elements to be culled by each client.

[0114] In this embodiment, when it is determined that cache culling is required by periodically monitoring the usage cache of the MDS, if the most recent monitoring cycle in which the cache is culled is the first monitoring cycle in which the cache is culled, the total number of cache elements to be culled determined in the most recent monitoring cycle in which the cache is culled is the sum of the reference number of culled elements configured for each client; if the most recent monitoring cycle in which the cache is culled is not the first monitoring cycle in which the cache is culled, the total number of cache elements to be culled in the current monitoring cycle is determined based on the change in the usage cache within two consecutive preset time periods and the total number of cache elements to be culled determined in the most recent monitoring cycle in which the cache is culled; then the number of sub-cache elements to be culled for each client is determined based on the determined total number of cache elements, as well as the weight and reference number of culled elements configured for each client, and cache culling is performed according to the determined number of sub-cache elements to be culled for each client. When determining the number of sub-cache elements to be removed for each client in the current monitoring cycle, this solution considers the cache changes in two consecutive preset time periods, the total number of cache elements to be removed in the most recent monitoring cycle that removed the cache, and the weight and reference number of removed elements configured for each client, so that cache removal can be performed dynamically, accurately, and in combination with the situation of each client. Therefore, this solution can improve the management capabilities of MDS and enhance customer experience.

[0115] In another example, in step 202 and step 304, the cache change difference is calculated according to the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time and the change amount of the used cache of the metadata management component in the previous preset time period, including:

[0116] Calculate the change in the usage cache of the metadata management component within a preset time period corresponding to the current time;

[0117] Calculate the change in the usage cache of the metadata management component within the last preset time period;

[0118] Calculate the difference between the change amount of the metadata management component's cache usage in the preset time period corresponding to the current time and the change amount of the metadata management component's cache usage in the previous preset time period as the cache change difference corresponding to the current monitoring period;

[0119] The calculation of the change in the usage cache of the metadata management component within a preset time period includes:

[0120] Three sub-time intervals are set for the preset time period, namely, a first sub-time interval, a second sub-time interval and a third sub-time interval; wherein the first sub-time interval covers the second sub-time interval, and the second sub-time interval covers the third sub-time interval;

[0121] Calculate the cache changes per second in the three sub-time periods respectively;

[0122] The weighted sum of the cache changes per second in the three sub-time periods is used to obtain the change in the cache usage of the metadata management component within the preset time period;

[0123] When it is determined based on the comparison result of the cache change difference calculated in the current monitoring period and 0 that the total number of cache elements to be eliminated in the current monitoring period is N times the total number of cache elements to be eliminated determined in the most recent monitoring period in which the cache is eliminated, where N is 0, 1 or greater than 1, specifically:

[0124] Compare the cache change difference with the size of 0, and determine whether the cache is growing;

[0125] If the cache change difference is greater than 0, then N is greater than 1;

[0126] If the cache change difference is equal to 0, and the cache usage is in a growth state, then N is greater than 1;

[0127] If the cache change difference is equal to 0, and the cache usage is not in the growth state, N is 0;

[0128] If the cache change difference is less than 0, N is 1.

[0129] In step 204 and step 306, the number of sub-cache elements to be removed from each client is determined according to the total number of cache elements to be removed in the current monitoring cycle, and the weight configured for each client and the reference number of removed elements, including:

[0130] Determine the estimated number of sub-cache elements to be removed for each client based on the total number of cache elements to be removed in the current monitoring cycle and the weight configured for each client;

[0131] The estimated number of sub-cache elements to be removed corresponding to each client is compared with the reference number of removed elements configured for each client, and the larger number of cache elements is determined as the actual number of sub-cache elements to be removed corresponding to the client.

[0132] The following is a detailed description of the calculation process of the cache change difference, the total number of cache elements to be removed in the current monitoring cycle, and the number of sub-cache elements to be removed by each client, in conjunction with the accompanying drawings.

[0133] The preset time period is usually divided into multiple sub-time periods, and the cache changes within a period are counted respectively, which is used to count the change trend of cache usage within multiple sub-time periods, and then calculate the cache change difference between two preset time periods. The number of sub-time periods and the length of each sub-time period can be flexibly set according to needs.

[0134] In the embodiment of the present application, division into three sub-time intervals is taken as an example.

[0135] In the embodiment of the present application, when calculating the cache change difference, it is necessary to first calculate the change in the cache usage of the data management component in two consecutive preset time periods. The implementation method of calculating the change in the cache usage of the data management component in each preset time period is the same. First, three sub-time period intervals are set for the preset time period, namely, the first sub-time period interval, the second sub-time period interval and the third sub-time period interval; wherein the first sub-time period interval covers the second sub-time period interval, and the second sub-time period interval covers the third sub-time period interval.

[0136] See also Figure 4 , Figure 4 The following is a schematic diagram of three sub-time periods in a preset time period in the embodiment of the present application. Taking the preset time period as 60 minutes as an example, the three sub-time periods are: 60 minutes, 30 minutes and 5 minutes, and the sub-time periods are determined by pushing back from the current time.

[0137] Calculate the change in the cache usage of the data management component within each preset time period, specifically:

[0138] The first step is to calculate the cache change per second in the three sub-time periods respectively;

[0139] like Figure 4 In the example, the cache change per second in the first sub-time period is D(60)=(ZA) / (60×60); the cache change per second in the second sub-time period is D(30)=(ZB) / (30×60); and the cache change per second in the third sub-time period is D(5)=(ZC) / (5×60).

[0140] The second step is to perform a weighted summation of the cache changes per second in the three sub-time periods to obtain the cache changes used by the metadata management component within the preset time period.

[0141] Combination Figure 4 , calculate the change in the usage cache of the metadata management component within the preset time period as:

[0142] D(60)×W(60)+D(30)×W(30)+D(5)×W(5);

[0143] Wherein, W(60) is the weight of the first sub-time interval, W(30) is the weight of the second sub-time interval, and W(5) is the weight of the third sub-time interval.

[0144] The maximum time interval 60 represents the cache change trend in the recent period. Usually, the sensitivity to cache changes is not high, that is, the sensitivity to sudden changes in the cache is low, which helps to balance occasional instantaneous traffic and avoid misjudgment of the overall cache change trend;

[0145] The smallest time interval 5 represents the cache change trend in the recent period. It is usually very sensitive to cache changes. That is to say, once there is a sudden change in the cache, the value calculated in this interval will be reflected. Therefore, the weight of this interval is usually high so that the overall cache change can be obtained. However, in order to prevent interference caused by occasional instantaneous traffic, it is necessary to use the calculated value of a larger interval for balance.

[0146] For example, the weights set for the first sub-time period, the second sub-time period, and the third sub-time period may be 0.1, 0.3, and 0.6, respectively, and may be adjusted according to actual needs, without limitation thereto.

[0147] Figure 4 The length and number of the sub-time periods given in the figure, as well as the weight settings, can be freely set, and are usually set specifically according to the business scenario. The division of the three sub-time periods and the setting of the length of each time period are only an example, but are not limited to this implementation.

[0148] See also Figure 5 , Figure 5 This is a schematic diagram of the process of calculating the cache change difference in the embodiment of the present application. The specific steps include:

[0149] Step 501: Calculate the change in the usage cache of the metadata management component within a preset time period corresponding to the current time.

[0150] The preset time period corresponding to the current time here is the time that is the longer of the preset time period starting from the current time.

[0151] Step 502: Calculate the change in the usage cache of the metadata management component within the last preset time period.

[0152] The previous preset time period is a previous preset time period that is continuous with the current preset time period.

[0153] Step 503: Calculate the difference between the change in the used cache of the metadata management component in the preset time period corresponding to the current time and the change in the used cache of the metadata management component in the previous preset time period as the cache change difference corresponding to the current monitoring period.

[0154] The change in the usage cache of the metadata management component corresponding to the preset time period with the current time as the end point is recorded as the change in the usage cache of the metadata management component in the preset time period corresponding to the current time D(current), and the change in the usage cache of the metadata management component in the previous preset time period is recorded as D(last). The cache change difference corresponding to the current monitoring cycle is D(diff) =D(current) - D(last). The calculation of the change in the usage cache of the metadata management component in each of the above time periods and the cache change difference corresponding to the current monitoring cycle are both numerical subtractions. Therefore, the calculated value may be greater than 0, equal to 0, or less than 0.

[0155] See also Figure 6 , Figure 6 This is a flow chart of determining the total number of cache elements to be removed in the current monitoring cycle in an embodiment of the present application. Specifically, it includes:

[0156] Step 601, compare the cache change difference with the size of 0, and determine whether the cache usage is in a growing state.

[0157] Step 602: If the cache change difference is greater than 0, determine that the total number of cache elements to be removed in the current monitoring cycle is N times the total number of cache elements to be removed in the most recent monitoring cycle in which the cache was removed; where N is greater than 1. End this process. If the cache change difference is greater than 0, it can be regarded as a scenario in which the use of the MDS cache increases suddenly, and the total number of cache elements to be removed needs to be increased. See Figure 7 , Figure 7 FIG. 1 is a schematic diagram of a scenario in which a cache burst growth is used in an embodiment of the present application. Figure 7As shown, in this scenario, since the growth curve of cache usage is an exponential curve, the growth difference of cache usage becomes larger and larger, that is, D(60) < D(30) < D(5). As time goes by, the final calculated D(current) >D(last), that is, D(diff)> 0. In this case, it means that the number of elements in the cache currently removed is not enough to suppress the growth of cache usage. The growth of cache usage is still very rapid, and the number of elements removed still needs to be increased. Therefore, for this situation of D(diff)> 0, in an embodiment of the present application, the total number of cache elements to be removed in the current monitoring cycle is determined to be greater than 1 times the total number of cache elements to be removed in the most recent monitoring cycle in which the cache was removed, that is, the total number of cache elements removed in the monitoring cycle is increased, such as N is set to 2, but is not limited to this.

[0158] Step 603: If the cache change difference is 0 and the cache usage is determined to be growing, determine that the total number of cache elements to be removed in the current monitoring cycle is N times the total number of cache elements to be removed in the most recent monitoring cycle in which the cache was removed, where N is greater than 1. End this process.

[0159] If the cache change difference is equal to 0, and the cache usage is determined to be growing, then this is a scenario where the memory is growing steadily, and the total number of cache elements to be culled needs to be increased. Figure 8 , Figure 8 This is a schematic diagram of a scenario in which a cache is steadily increased in accordance with an embodiment of the present application. Figure 8 As shown in the figure, in this scenario, since the memory growth curve is a linear curve, the memory growth difference remains consistent, that is, D(60), D(30), and D(5) are all equal, so the final calculated D(current) and D(last) are also equal, that is, D(diff) is 0. Therefore, in this case, it also means that the number of elements in the current cache that are eliminated is not enough to suppress the growth of memory, and the total number of cache elements that are eliminated still needs to be increased.

[0160] Step 604: If the cache change difference is equal to 0, and it is determined that the used cache is not in a growing state, then the total number of cache elements to be removed in the current monitoring cycle is determined to be 0. This process ends.

[0161] If the cache change difference is equal to 0 and the cache has not grown, it is a scenario where the cache is stable and unchanged, and cache eviction is not required in this scenario. Fig. 9 , Fig. 9 This is a schematic diagram of a scenario in which a cache is used to maintain a constant level in an embodiment of the present application. Fig. 9As shown in the figure, in this scenario, since the cache usage remains almost unchanged, the difference in cache usage is almost zero, that is, D(60) = D(30) = D(5) = 0. In this scenario, since the cache usage has been controlled and there is no increase in cache usage, cache eviction is no longer required. This can maximize the use of the current cache usage and avoid resource waste caused by not using the cache.

[0162] Step 605 , if the cache change difference is less than 0, determine that the total number of cache elements to be removed in the current monitoring cycle is the total number of cache elements to be removed determined in the most recent monitoring cycle in which the cache was removed.

[0163] If the cache change difference is less than 0, it is a scene where the MDS cache usage growth slows down, so the scene can keep the total number of cache elements to be culled unchanged. Fig.10 , Fig.10 Schematic diagram of a scenario in which cache growth is slowed down in an embodiment of the present application. Fig.10 As shown in the figure, in this scenario, since the growth curve of cache usage is a logarithmic curve, the growth difference of cache usage becomes smaller and smaller, that is, D(60) > D(30) > D(5). As time goes by, the final calculated D(current) < D(last), that is, D(diff) < 0. Therefore, in this case, it means that the number of elements currently removed can control the growth of the current cache usage, and the total number of cache elements removed can be kept unchanged, thereby gradually reducing the cache usage.

[0164] This embodiment takes into account the cache change difference in different preset time periods, and uses the current state of the cache to determine the total number of cache elements to be removed in the current monitoring cycle. By introducing a strategy for dynamically changing the number of total cache elements to be removed, in order to avoid the unreasonable configuration of the fixed removal strategy and the risk of causing the MDS component to use the cache over-limit alarm or even downtime, the MDS cache removal speed is dynamically adjusted by calculating the cache growth trend to ensure the MDS component's control over the cache limit.

[0165] In order to avoid a slight change in the use of the cache resulting in a large difference in the determination of the total number of cache elements, the cache change difference may be expanded to near 0 when it is 0, such as [0-T, 0+T], where T is an amplitude value set to avoid jitter.

[0166] See also Fig.11 , Fig.11 This is a flow chart of determining the number of sub-cache elements to be removed by each client in an embodiment of the present application. The specific steps are:

[0167] Step 1101 , determining the estimated number of sub-cache elements to be removed from each client according to the determined total number of cache elements to be removed in the current monitoring cycle and the weight configured for each client.

[0168] Step 1102, compare the estimated number of sub-cache elements to be removed corresponding to each client with the reference number of removed elements configured for each client, and determine the larger number of cache elements as the actual number of sub-cache elements to be removed corresponding to the client.

[0169] Here, the maximum value of the two is taken, so that the number of sub-cache elements to be culled by each client each time is not less than the number of reference culled elements configured for the client.

[0170] In this embodiment, fine-grained culling strategy management is achieved by setting the basic number of culling elements at the client granularity, that is, the number of reference culling elements, and the weight of each client, so as to facilitate personalized adaptation for different usage postures of different services.

[0171] In the embodiment of the present application, for different clients, when the used cache of the MDS is greater than the first cache threshold and less than the second cache threshold, the same dynamic cache removal strategy is configured; when the used cache of the MDS is not less than the second cache threshold, the same or different execution logic is configured. Fig.12 , Fig.12 This is a schematic diagram of a processing flow when the usage cache of the MDS is not less than the second cache threshold in an embodiment of the present application. The specific steps are:

[0172] Step 1201: when the used cache of the metadata management component monitored in the current monitoring period is not less than a second cache threshold, if a session request sent by a client is received, determining the execution logic configured for the client.

[0173] Step 1202: If it is determined that the execution logic configured for the client fails, the client replies with a message indicating that the operation failed due to a cache limit exceeding, and this process ends.

[0174] Step 1203: If it is determined that the execution logic configured for the client is waiting, the client is blocked and waits for a preset time.

[0175] Step 1204: If the preset time has expired and the current metadata management component's used cache is not less than the second cache threshold, a message is sent to the client indicating that the operation failed due to a waiting timeout, and this process ends.

[0176] Step 1205: If the used cache of the metadata management component is lower than the second cache threshold within the preset time, the session request sent by the client is processed and a processing result is responded to, and this process ends.

[0177] Step 1206: If the execution logic configured for the client is passed, skip the cache and process the session request sent by the client through the backend file, and respond with the processing result.

[0178] In the specific embodiment of the present application, a dynamic cache eviction strategy is provided which is triggered when the cache usage exceeds a first cache threshold. In order to solve the problem of continuous cache growth caused by a large number of requests that cannot be handled by a fixed eviction configuration, after eviction of a certain number of elements, the current cache usage will continue to be monitored to see if it is under control. If it is not controlled, the number of elements to be evicted next time will be gradually increased. Through this strategy of dynamically increasing the eviction elements, the cache usage is further controlled. If the final cache usage still reaches the second cache threshold, the solution can also process the behavior of the corresponding client separately according to the execution logic configured for different clients, and further force the cache usage to be reduced below the first cache threshold, thereby ensuring that the usage of cache resources is controlled.

[0179] This solution solves the problem that cache culling rules in related solutions are relatively simple and lack flexibility. The present invention introduces client-level culling configuration and supports weight settings between clients, so that different culling numbers and weights can be configured according to the usage posture and business needs of each client, and the number of elements to be culled by each client when the cache needs to be culled can be dynamically adjusted according to the changing trend of the cache usage. Under the premise of ensuring that the use of the cache is controllable, the performance requirements of business access requests are also guaranteed, thereby improving the management capabilities of MDS and the client experience.

[0180] Based on the same inventive concept, the embodiment of the present application also provides a cache management device for a metadata management component. Fig.13 , Fig.13 This is a schematic diagram of the structure of a cache management device of a metadata management component in an embodiment of the present application. The device includes:

[0181] A monitoring unit 1301 is used to periodically monitor the usage cache of the metadata management component;

[0182] The calculation unit 1302 is configured to calculate a cache change difference value according to a change amount of the used cache of the metadata management component in a preset time period corresponding to the current time and a change amount of the used cache of the metadata management component in a previous preset time period in response to the used cache of the metadata management component monitored in the current monitoring period being greater than a first cache threshold and less than a second cache threshold; wherein the first cache threshold is less than the second cache threshold;

[0183] The determining unit 1303 is used to determine the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; and determine the number of sub-cache elements to be eliminated for each client according to the determined total number of cache elements to be eliminated in the current monitoring cycle, and the weight configured for each client and the reference number of eliminated elements;

[0184] A processing unit 1304 is used to perform cache culling according to the determined number of sub-cache elements to be culled by each client;

[0185] Among them, the determination unit is specifically used to determine, based on the comparison result of the cache change difference calculated in the current monitoring cycle and 0, whether the total number of cache elements to be eliminated in the current monitoring cycle is N times the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; wherein N is 0, 1 or a value greater than 1; determine the estimated number of sub-cache elements to be eliminated for each client based on the determined total number of cache elements to be eliminated in the current monitoring cycle and the weight configured for each client; compare the estimated number of sub-cache elements to be eliminated corresponding to each client with the reference number of eliminated elements configured for each client, and the number of cache elements with a larger value is determined as the actual number of sub-cache elements to be eliminated corresponding to the client.

[0186] In another example,

[0187] The calculation unit 1302 is further used to determine whether the cache of two consecutive preset time periods has been used, and whether the current monitoring cycle is the first monitoring cycle to remove the cache, in response to the use of the cache of the metadata management component monitored in the current monitoring period being greater than the first cache threshold and less than the second cache threshold; if the cache of two consecutive preset time periods has been used, and the current monitoring cycle is not the first monitoring cycle to remove the cache, then calculate the cache change difference corresponding to the current monitoring cycle based on the change of the use of the cache of the metadata management component in the preset time period corresponding to the current time, and the change of the use of the cache of the metadata management component in the previous preset time period; otherwise, determine that the reference number of culling elements configured for each client is the number of sub-cache elements to be removed by the client; and trigger the processing unit 1304 to perform cache culling according to the determined number of sub-cache elements to be removed by each client.

[0188] In another example,

[0189] The calculation unit 1302 is specifically configured to calculate the cache change difference according to the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time and the change amount of the used cache of the metadata management component in the previous preset time period, calculate the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time; calculate the change amount of the used cache of the metadata management component in the previous preset time period; calculate the difference between the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time and the change amount of the used cache of the metadata management component in the previous preset time period as the cache change difference corresponding to the current monitoring period;

[0190] in,

[0191] Calculate the change in the usage cache of the metadata management component within a preset time period, including:

[0192] Three sub-time intervals are set for the preset time period, namely, a first sub-time interval, a second sub-time interval and a third sub-time interval; wherein the first sub-time interval covers the second sub-time interval, and the second sub-time interval covers the third sub-time interval;

[0193] Calculate the cache changes per second in the three sub-time periods respectively;

[0194] The weighted sum of the cache changes per second in the three sub-time periods is used to obtain the change in the cache usage of the metadata management component within the preset time period.

[0195] In another example,

[0196] Calculation unit 1302 is specifically used to compare the size of the cache change difference and 0, and determine whether the cache usage is in a growing state; if the cache change difference is greater than 0, N is greater than 1; if the cache change difference is equal to 0, and the cache usage is in a growing state, N is greater than 1; if the cache change difference is equal to 0, and the cache usage is not in a growing state, N is 0; if the cache change difference is less than 0, N is 1.

[0197] In another example,

[0198] Processing unit 1304 is further used to reply to the client that sent the session request with a message that the operation failed due to cache limit if a session request sent by the client is received and the execution logic configured for the client is failed, when the usage cache of the metadata management component monitored in the current monitoring period is not less than a second cache threshold.

[0199] In another example,

[0200] The processing unit 1304 is further configured to block the client and wait for a preset time if a session request sent by a client is received and the execution logic configured for the client is to wait when the usage cache of the metadata management component monitored in the current monitoring period is not less than a second cache threshold;

[0201] If the preset time is reached and the used cache of the current metadata management component is not less than the second cache threshold, a message is replied to the client indicating that the operation failed due to waiting timeout;

[0202] If the used cache of the metadata management component is lower than the second cache threshold within the preset time, the session request sent by the client is processed and the processing result is responded.

[0203] In another example,

[0204] Processing unit 1304 is further used to skip the cache and process the session request sent by the client through the backend file when the usage cache of the metadata management component monitored in the current monitoring cycle is not less than the second cache threshold, if a session request sent by the client is received and the execution logic configured for the client is passed, and respond to the processing result.

[0205] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a cache management method of a metadata management component when executing the program.

[0206] In another embodiment, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, a cache management method of a metadata management component is implemented.

[0207] Fig.14 Schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. Fig.14 As shown, the electronic device may include: a processor (Processor) 1410, a communication interface (Communications Interface) 1420, a memory (Memory) 1430 and a communication bus 1440, wherein the processor 1410, the communication interface 1420, and the memory 1430 communicate with each other through the communication bus 1440. The processor 1410 may call the logic instructions in the memory 1430 to execute the following method:

[0208] Periodically monitor the usage cache of metadata management components;

[0209] In response to the use cache of the metadata management component monitored in the current monitoring period being greater than a first cache threshold and less than a second cache threshold, a cache change difference is calculated according to a change amount of the use cache of the metadata management component in a preset time period corresponding to the current time and a change amount of the use cache of the metadata management component in a previous preset time period; wherein the first cache threshold is less than the second cache threshold;

[0210] Determine the total number of cache elements to be culled in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be culled determined in the most recent monitoring cycle in which the cache was culled;

[0211] Determine the number of sub-cache elements to be removed from each client according to the total number of cache elements to be removed in the current monitoring cycle, the weight configured for each client, and the reference number of removed elements;

[0212] Cache culling is performed according to the number of sub-cache elements to be culled for each client;

[0213] The total number of cache elements to be removed in the current monitoring cycle is determined based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be removed determined in the most recent monitoring cycle in which the cache was removed, including:

[0214] Based on the comparison result of the cache change difference calculated in the current monitoring period and 0, determine that the total number of cache elements to be eliminated in the current monitoring period is N times the total number of cache elements to be eliminated determined in the most recent monitoring period in which the cache was eliminated; where N is 0, 1 or greater than 1;

[0215] The number of sub-cache elements to be removed from each client is determined based on the total number of cache elements to be removed in the current monitoring cycle, as well as the weight and reference number of removed elements configured for each client, including:

[0216] Determine the estimated number of sub-cache elements to be removed for each client based on the total number of cache elements to be removed in the current monitoring cycle and the weight configured for each client;

[0217] The estimated number of sub-cache elements to be removed corresponding to each client is compared with the reference number of removed elements configured for each client, and the larger number of cache elements is determined as the actual number of sub-cache elements to be removed corresponding to the client.

[0218] In addition, the logic instructions in the above-mentioned memory 1430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0219] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0220] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0221] The flow chart and block diagram in the accompanying drawings of the present application show the possible architecture, function and operation of the system, method and computer program product according to the various embodiments disclosed in the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0222] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims disclosed in this application may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments and / or claims of this application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope disclosed in this application.

[0223] Specific embodiments are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas, and is not used to limit the present application. For those skilled in the art, changes can be made in the specific implementation methods and application scopes according to the ideas, spirits and principles of the present invention, and any modifications, equivalent substitutions, improvements, etc. made therein should be included in the scope of protection of this application.

Claims

1. A cache management method for a metadata management component, characterized in that: The method comprises: Periodically monitor the usage cache of metadata management components; In response to the use cache of the metadata management component monitored in the current monitoring period being greater than a first cache threshold and less than a second cache threshold, a cache change difference is calculated according to a change amount of the use cache of the metadata management component in a preset time period corresponding to the current time, and a change amount of the use cache of the metadata management component in a previous preset time period; wherein the first cache threshold is less than the second cache threshold; Determine the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; Determine the number of sub-cache elements to be removed from each client according to the total number of cache elements to be removed in the current monitoring cycle, as well as the weight configured for each client and the reference number of removed elements; Perform cache culling according to the determined number of sub-cache elements to be culled by each client; The determining of the total number of cache elements to be removed in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be removed determined in the most recent monitoring cycle in which the cache was removed includes: Based on the comparison result of the cache change difference calculated in the current monitoring cycle and 0, determine that the total number of cache elements to be eliminated in the current monitoring cycle is N times the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; wherein N is 0, 1 or greater than 1; The determining the number of sub-cache elements to be removed from each client according to the determined total number of cache elements to be removed in the current monitoring cycle, and the weight configured for each client and the reference number of removed elements, includes: Determining the estimated number of sub-cache elements to be removed for each client according to the determined total number of cache elements to be removed in the current monitoring cycle and the weight configured for each client; The estimated number of sub-cache elements to be removed corresponding to each of the clients is compared with the reference number of removed elements configured for each of the clients, and the larger number of cache elements is determined as the actual number of sub-cache elements to be removed corresponding to the client.

2. The method according to claim 1, characterized in that After the used cache of the metadata management component monitored in the current monitoring period is greater than the first cache threshold and less than the second cache threshold, before calculating the cache change difference corresponding to the current monitoring period according to the change of the used cache of the metadata management component in the preset time period corresponding to the current time and the change of the used cache of the metadata management component in the previous preset time period, the method further includes: Determine whether the cache has been used for two consecutive preset time periods and whether the current monitoring cycle is the first monitoring cycle to evict the cache; If the cache has been used for two consecutive preset time periods and the current monitoring period is not the first monitoring period for which the cache is to be removed, the step of calculating the cache change difference corresponding to the current monitoring period based on the change in the use of the cache of the metadata management component within the preset time period corresponding to the current time and the change in the use of the cache of the metadata management component within the previous preset time period is performed; otherwise, the reference number of removal elements configured for each client is determined to be the number of sub-cache elements to be removed by the client, and cache removal is performed according to the determined number of sub-cache elements to be removed by each client.

3. The method according to claim 1, characterized in that The method further comprises: Calculating the change in the usage cache of the metadata management component within the preset time period includes: Three sub-time intervals are set for the preset time period, namely a first sub-time interval, a second sub-time interval and a third sub-time interval; wherein the first sub-time interval covers the second sub-time interval, and the second sub-time interval covers the third sub-time interval; Calculate the cache change per second in the three sub-time periods respectively; The weighted sum of the cache changes per second in the three sub-time periods is used to obtain the change in the cache usage of the metadata management component within the preset time period; The calculating the cache change difference according to the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time and the change amount of the used cache of the metadata management component in the previous preset time period includes: Calculate the change in the usage cache of the metadata management component within a preset time period corresponding to the current time; Calculate the change in the usage cache of the metadata management component within the previous preset time period; The difference between the change amount of the used cache of the metadata management component in the preset time period corresponding to the current time and the change amount of the used cache of the metadata management component in the previous preset time period is calculated as the cache change difference corresponding to the current monitoring period.

4. The method according to claim 1, characterized in that: The comparison result of the cache change difference calculated based on the current monitoring period and 0 determines that the total number of cache elements to be eliminated in the current monitoring period is N times the total number of cache elements to be eliminated determined in the most recent monitoring period in which the cache was eliminated; wherein N is 0, 1 or greater than 1, including: Compare the cache change difference with the size of 0, and determine whether the cache is growing; If the cache change difference is greater than 0, then N is greater than 1; If the cache change difference is equal to 0, and the cache usage is in a growth state, then N is greater than 1; If the cache change difference is equal to 0, and the cache usage is not in the growth state, N is 0; If the cache change difference is less than 0, N is 1.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the usage cache of the metadata management component monitored in the current monitoring period is not less than the second cache threshold, if a session request sent by a client is received and the execution logic configured for the client is failed, a message that the operation failed due to cache excess is replied to the client that sent the session request.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the usage cache of the metadata management component monitored in the current monitoring cycle is not less than the second cache threshold, if a session request sent by a client is received and the execution logic configured for the client is to wait, the client is blocked and waits for a preset time; If the preset time is reached and the current cache usage of the metadata management component is not less than the second cache threshold, a message is replied to the client indicating that the operation failed due to a waiting timeout; If the used cache of the metadata management component is lower than the second cache threshold within a preset time, the session request sent by the client is processed and a processing result is responded.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the usage cache of the metadata management component monitored in the current monitoring cycle is not less than the second cache threshold, if a session request sent by the client is received and the execution logic configured for the client is passed, the cache is skipped and the session request sent by the client is processed through the back-end file, and the processing result is responded.

8. A cache management device for a metadata management component, characterized in that: The device comprises: A monitoring unit, used for periodically monitoring the usage cache of the metadata management component; a calculation unit, configured to calculate a cache change difference according to a change amount of the used cache of the metadata management component in a preset time period corresponding to the current time and a change amount of the used cache of the metadata management component in a previous preset time period in response to the used cache of the metadata management component monitored in the current monitoring period being greater than a first cache threshold and less than a second cache threshold; wherein the first cache threshold is less than the second cache threshold; A determination unit, configured to determine the total number of cache elements to be eliminated in the current monitoring cycle based on the cache change difference calculated in the current monitoring cycle and the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; and determine the number of sub-cache elements to be eliminated for each client based on the total number of cache elements to be eliminated in the current monitoring cycle determined, and the weight configured for each client and the reference number of eliminated elements; A processing unit, configured to perform cache culling according to the determined number of sub-cache elements to be culled by each client; Among them, the determination unit is specifically used to determine, based on the comparison result of the cache change difference calculated in the current monitoring cycle and 0, whether the total number of cache elements to be eliminated in the current monitoring cycle is N times the total number of cache elements to be eliminated determined in the most recent monitoring cycle in which the cache was eliminated; wherein N is 0, 1 or a value greater than 1; determine the estimated number of sub-cache elements to be eliminated for each client based on the determined total number of cache elements to be eliminated in the current monitoring cycle and the weight configured for each client; compare the estimated number of sub-cache elements to be eliminated corresponding to each client with the reference number of elimination elements configured for each client, and determine the larger value of the number of cache elements as the actual number of sub-cache elements to be eliminated corresponding to the client.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 7 is implemented.

Images