Automatic Alluxio nested cache expelling method and storage medium

Abstract

The invention provides an Alluxio nested cache automatic expelling method and a storage medium, the method comprises a database establishment step, an expelling rule issuing step, an expelling rule updating step and a cache expelling management step, a directory-level TTL rule is adopted, the TTL rule is stored in an etcd in the form of a key value pair of a directory path and TTL expelling duration, and the TTL expelling duration is stored in the etcd in the form of a key value pair of the directory path and the TTL expelling duration. Synchronizing to all Worker nodes through a heartbeat mechanism; through a timed scanning mechanism, the system regularly checks the survival time of cached data and automatically expels overtime data; and finally, dynamic adjustment of the TTL rule is supported, a user can flexibly add, update or delete the rule through a CLI command, and the rule change can be quickly responded. The method has the advantages that the complexity of cache management is reduced, the data access performance and the system response efficiency are improved, and the method is suitable for big data and distributed computing environments.

Description

Technical Field

[0001] This application relates to the technical field of distributed cache management, and specifically relates to an Alluxio nested cache automatic eviction method and a storage medium. Background Art

[0002] As a distributed cache system, Alluxio supports a variety of different cache eviction strategies. A relatively common strategy is the cache eviction strategy based on TTL.

[0003] The traditional TTL-based cache eviction strategy (Time-to-Live, TTL-based cache eviction strategy) is an algorithm for cache management (hereinafter referred to as the "TTL strategy"), which relies on setting a specific survival time (TTL) for each entry in the cache. When the survival time of an entry exceeds the predefined TTL value, whether it is accessed or not, the entry will be automatically deleted or marked as expired. This can effectively prevent the data in the cache from becoming obsolete, save memory, and ensure that the data in the cache is relatively new.

[0004] However, due to the following disadvantages of the traditional TTL method: lack of precise control over the data life cycle, inability to effectively handle data with a fixed life cycle; complex cache management, frequent eviction operations increase the system complexity and reduce the cache efficiency; inability to adapt to large-scale distributed environments, and difficulty in coping with the management requirements of billions of file metadata; and Alluxio needs to support the management of billions of file metadata. Therefore, it is not realistic to set a TTL rule for each file. Summary of the Invention

[0005] This application provides an Alluxio nested cache automatic eviction method and a storage medium to solve the problem that the existing TTL-based cache eviction strategy cannot support the setting of directory-level TTL rules.

[0006] This application provides an Alluxio nested cache automatic eviction method, including a database establishment step, an eviction rule publishing step, an eviction rule updating step, and a cache eviction management step.

[0007] The database establishment step is used to establish a distributed key-value database for storing a cache eviction rule set. The cache eviction rule set includes at least one cache eviction rule, and the cache eviction rule is stored in the form of a key-value pair composed of a directory path and a TTL eviction duration; the eviction rule publishing step is that the user publishes different eviction rule commands through a command line interface CLI; the eviction rule update step is based on the eviction rule command, and each Alluxio Worker node periodically interacts with the distributed key-value database through heartbeats to update the cache eviction rule set; the cache eviction management step is used to periodically check the data in each cache page in the local file system and clear the timed-out cache pages, and at least one directory path is included in the cache pages.

[0008] Further, when the eviction rule command is an add eviction rule command, the eviction rule publishing step includes an eviction rule construction step, a legal verification step, and an eviction rule specification step.

[0009] The eviction rule construction step is used to construct an add eviction rule; the legal verification step is used to judge whether the directory path of the add eviction rule is correct and whether the TTL eviction duration of the add eviction rule is legal. If so, execute the next step; if not, publish an exception notice; the eviction rule specification step is used to standardize the directory path and TTL eviction duration of the add eviction rule to form a standardized eviction rule, and store the standardized eviction rule in the distributed key-value database.

[0010] Further, when the eviction rule command is a delete eviction rule command, the eviction rule publishing step includes a directory path verification step and an eviction rule deletion step.

[0011] The directory path verification step is used to judge whether the directory path corresponding to the eviction rule to be deleted exists. If so, execute the next step; if not, publish an exception notice; the eviction rule deletion step is used to delete the eviction rule to be deleted.

[0012] Further, when the eviction rule command is an update eviction rule command, the eviction rule publishing step includes an eviction rule verification step and an eviction rule update step.

[0013] The eviction rule verification step is used to judge whether the eviction rule to be updated exists. If so, execute the next step; if not, publish an exception notice; the eviction rule update step is to configure a new TTL eviction duration for the directory path corresponding to the eviction rule to be updated, and update the eviction rule to be updated.

[0014] Further, when the eviction rule command is a view eviction rule command, the directory paths and TTL eviction durations of each eviction rule in the cache eviction rule set are displayed in a structured format.

[0015] Further, the eviction rule update step includes a command judgment step and a command discrimination step.

[0016] In the command judgment step, each Alluxio Worker node periodically performs heartbeat interaction with the distributed key-value database. The Alluxio Worker node pulls the cache eviction rule set from the distributed key-value database and judges whether the eviction rule command is legal based on the cache eviction rule set. If so, the next step is executed; if not, an exception notice is issued. The command discrimination step is used to judge whether the eviction rule command is a view eviction rule command. If so, the latest cache eviction rule is viewed; if not, the cache eviction rule set is updated according to the eviction rule command to obtain an updated cache eviction rule set.

[0017] Further, the cache eviction management step includes a cache page scanning step, a cache page judgment step, an eviction rule lookup step, a cache page time judgment step, and a cache page eviction step.

[0018] The cache page scanning step is used to create a background scanning program to periodically scan all cache pages. The cache page judgment step is used to judge whether there are un-scanned cache pages. If so, the next step is executed; if not, the scanning ends. The eviction rule lookup step is based on the longest prefix matching rule to find whether the directory path in the un-scanned cache page can match the directory path corresponding to the eviction rule in the updated cache eviction rule. If not, it returns to the cache page judgment step; if so, the eviction rule in the updated cache eviction rule that the directory path in the un-scanned cache page can match is defined as the matching eviction rule, and the next step is executed. The cache page time judgment step is based on the matching eviction rule to judge whether the un-scanned cache page has expired. If not, it returns to the cache page judgment step; if so, the next step is executed. The cache page eviction step is used to evict the un-scanned cache page and judge whether the eviction is successful. If so, it returns to the cache page judgment step; if not, an error information log is recorded.

[0019] The present invention also provides a storage medium storing computer-readable instructions, which, when read by at least one processor, cause the at least one processor to execute at least one step in the Alluxio nested cache automatic eviction method described above.

[0020] This application provides an Alluxio nested cache automatic eviction method and a storage medium. First, a directory-level TTL rule is adopted, and the TTL rule is stored in etcd in the form of a key-value pair of directory path and TTL eviction duration, and is synchronized to all Worker nodes through a heartbeat mechanism, simplifying the management of large-scale file metadata. Second, through a timed scanning mechanism, the system regularly checks the survival time of cached data and automatically evicts expired data to ensure that only valid data is retained in the cache, improving the utilization rate of the cache space. Finally, it supports dynamic adjustment of TTL rules. Users can flexibly add, update, or delete rules through CLI commands, and the system can quickly respond to rule changes to meet the requirements of different application scenarios. These technical means significantly reduce the complexity of cache management, reduce the system burden, and at the same time improve data access performance and system response efficiency, and are particularly suitable for big data and distributed computing environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0022] Figure 1 is the flowchart of the Alluxio nested cache automatic eviction method described in the embodiments of the present application; Figure 2 is the flowchart of the eviction rule publishing step described in the embodiments of the present application Figure 1 ; Figure 3 is the flowchart of the eviction rule publishing step described in the embodiments of the present application Figure 2 ; Figure 4 is the flowchart of the eviction rule publishing step described in the embodiments of the present application Figure 3 ; Figure 5 is the flowchart of the eviction rule update step described in the embodiments of the present application; Figure 6 is the flowchart of the cache eviction management step described in the embodiments of the present application; Figure 7 is the schematic diagram of the storage medium described in the embodiments of the present application.

[0023] DESCRIPTION OF REFERENCE NUMERALS: 100 storage medium, 110 processor, 120 memory. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0025] As Figure 1 shown, the present application provides an Alluxio nested cache automatic eviction method, including step S1) database establishment step, step S2) eviction rule publishing step, step S3) eviction rule updating step, and step S4) cache eviction management step.

[0026] Step S1) Database establishment step: Establish a distributed key-value database etcd to store a cache eviction rule set. The cache eviction rule set includes at least one cache eviction rule, and the cache eviction rule is stored in the form of a key-value pair composed of a directory path and a TTL eviction duration. This design facilitates the unified and flexible acquisition and update of the cache duration of different directory data by each Worker node in the Alluxio cluster.

[0027] In this embodiment, to ensure the efficiency of rule storage, a hierarchical storage structure is designed when organizing the rules. Namespaces or groups are generated according to different directory paths, which is convenient for quickly searching and updating rules for specific paths in a complex cache rule system. With this design, a rule management service with low latency and high stability can be provided in a high-concurrency environment.

[0028] Step S2) Eviction rule publishing step: The user publishes different eviction rule commands through a command-line interface CLI.

[0029] In this embodiment, a convenient command-line interface (CLI) is provided for the user to realize the interaction of the client. The user can flexibly perform management tasks such as adding, deleting, updating, and viewing cache eviction rules through simple and intuitive command operations. This design takes into account the user experience and operation efficiency, ensuring that the system is easy to understand and manage during use.

[0030] As Figure 2 shown, when the eviction rule command is an add eviction rule command, step S2) eviction rule publishing step includes step S21) eviction rule construction step, step S22) legal verification step, and step S23) eviction rule specification step.

[0031] Step S21) Eviction rule construction step: Construct an add eviction rule.

[0032] Step S22) Legal verification step, to determine whether the directory path for adding the eviction rule is correct and whether the TTL eviction duration for adding the eviction rule is legal. If so, proceed to the next step; if not, issue an exception notice.

[0033] Step S23) Eviction rule specification step, to standardize the directory path and TTL eviction duration for adding the eviction rule, form a standardized eviction rule, and store the standardized eviction rule in the distributed key-value database.

[0034] In this embodiment, the user can add a new eviction rule to the system through the CLI command interface. This operation allows the user to specify an independent TTL eviction duration for different directories or data paths, that is, an eviction rule is constructed. After receiving the user's addition request, necessary parameter verification will be performed to ensure that the TTL eviction rule meets the format requirements. For example, the correctness of the directory path and the legality of the TTL eviction duration, etc. The judgment criteria for the correctness of the directory path and the legality of the TTL eviction duration can be set according to the user's needs and will not be elaborated here. If the parameters provided by the user are correct, the corresponding eviction rule will be constructed, that is, a standardized eviction rule in a format that can be stored in the distributed key-value database.

[0035] As Figure 3 shown, when the eviction rule command is a delete eviction rule command, step S2) Eviction rule publishing step includes step S24) Directory path verification step and step S25) Eviction rule deletion step.

[0036] Step S24) Directory path verification step, to determine whether the directory path corresponding to the eviction rule to be deleted exists. If so, proceed to the next step; if not, issue an exception notice.

[0037] Step S25) Eviction rule deletion step, to delete the eviction rule requested to be deleted.

[0038] In this embodiment, the user can delete the cache eviction rule in the existing cache eviction rule set through a command to release relevant resources when this rule is no longer applicable. After the CLI command interface receives the deletion request, it will perform a matching verification on the requested directory path and verify whether the rule exists. Through this design, it is ensured that the user only deletes valid rules and avoids the impact on the system caused by misoperations.

[0039] As Figure 4 shown, when the eviction rule command is an update eviction rule command, step S2 Eviction rule publishing step includes step S26) Eviction rule verification step and step S27) Eviction rule update step.

[0040] Step S26) Eviction rule verification step: Determine whether the eviction rule to be updated exists. If it does, proceed to the next step; if not, issue an exception notice.

[0041] Step S27) Eviction rule update step: Configure a new TTL eviction duration for the directory path corresponding to the eviction rule to be updated, and update the eviction rule to be updated.

[0042] In this embodiment, users are allowed to dynamically update the TTL eviction duration configuration of the cache eviction rule set. When a user initiates an update operation, the existence of the rule will be verified first, and then the rule content will be updated according to the newly set TTL duration. The updated rule is synchronized to etcd through the eviction rule maintenance module to ensure that all Worker nodes in the system can perform cache eviction according to the latest rules.

[0043] Furthermore, when the eviction rule command is a view eviction rule command, the directory path and TTL eviction duration of each eviction rule in the cache eviction rule set are displayed in a structured format to improve the readability of the data and facilitate users to quickly grasp the distribution of the current cache eviction policy.

[0044] In this embodiment, the latest cache eviction rule set is obtained by updating the cache eviction rule set through the eviction rule command.

[0045] Step S3) Eviction rule update step: Based on the eviction rule command, each Alluxio Worker node periodically performs heartbeat interaction with the distributed key-value database to update the cache eviction rule set.

[0046] As Figure 5 shown, Step S3) Eviction rule update step includes Step S31) Command judgment step and Step S32) Command discrimination step.

[0047] Step S31) Command judgment step: Each Alluxio Worker node periodically performs heartbeat interaction with the distributed key-value database. The Alluxio Worker node pulls the cache eviction rule set from the distributed key-value database and determines whether the eviction rule command is legal based on the cache eviction rule set. If it is, proceed to the next step; if not, issue an exception notice.

[0048] In this embodiment, the cache eviction rule set pulled by the Alluxio Worker node from the distributed key-value database is the latest cache eviction rule set.

[0049] Step S32) Command discrimination step: Determine whether the eviction rule command is a view eviction rule command. If so, view the latest cached eviction rule. If not, update the cached eviction rule set according to the eviction rule command to obtain an updated cached eviction rule set.

[0050] In this embodiment, by having each Alluxio Worker node perform heartbeat interactions with the distributed key-value database at regular intervals, centralized management, real-time update, and synchronous transmission of cached eviction rules are ensured. The main objective of this design is to efficiently store, update, and distribute cached eviction rules to each Worker node, so that the system can perform cache management based on the latest rules, enabling the Alluxio system to maintain dynamic consistency of the cache eviction policy in a multi-node environment and ensuring the validity of cached data in the system.

[0051] The heartbeat interaction sets a timing update mechanism that can periodically pull and obtain the latest cached eviction rules from etcd. This mechanism is based on the preset heartbeat time interval of the system to ensure that each Worker node can obtain the latest rules in a timely manner. A synchronization request is sent to etcd within the heartbeat cycle to cache the updated rules in etcd, which are then passed to the Worker nodes in the next cycle. This heartbeat interaction design ensures the consistency and real-time nature of rule information, enabling the cache system to respond quickly when rules change.

[0052] In this embodiment, to enhance the sensitivity to changes in cached eviction rules, a listening mechanism of etcd is also adopted. This mechanism will monitor in real time the changes in rules under the specified path. When a rule is added, deleted, or updated, the module will immediately trigger a change notification to obtain the latest rules in a timely manner. The change listening mechanism avoids the timeliness issues that may be caused by relying solely on periodic updates, ensuring that rule changes can be quickly synchronized to etcd and improving the system response efficiency.

[0053] When receiving a rule modification request sent by the CLI command interface, strict rule verification and exception handling will be performed to ensure that the updated TTL eviction rule meets the system requirements. The verification includes checking the directory path format, confirming the validity of the TTL eviction duration, etc. Error messages will be returned for configurations that do not meet the requirements and logs will be recorded. In addition, to handle the situation of network exceptions and etcd connection failures, a retry mechanism and an exception warning strategy are designed. By disconnecting and reconnecting and sending warning notifications, the robustness of the system is enhanced, enabling the cache management system to still operate effectively in an adverse environment.

[0054] Step S4) Cache Eviction Management Step: Periodically check the data in each cache page in the local file system and clear the cache pages that have timed out. Each cache page includes at least one directory path. By regularly scanning all cached data and evicting the timed-out data, the performance and responsiveness of the system are maintained.

[0055] In this embodiment, the latest TTL eviction rules are obtained from the distributed key-value database etcd, these rules are parsed, and applied to the management of each cached data page. When each cache page is added to the cache, its insertion time and the corresponding TTL eviction duration are recorded. Based on this information, the validity of each data item is judged to ensure that eviction processing can be carried out in a timely manner after timeout.

[0056] In this embodiment, all the data stored in the local file system is sliced into cache pages for management.

[0057] As Figure 6 shown, Step S4) Cache Eviction Management Step includes Step S41) Cache Page Scanning Step, Step S42) Cache Page Judgment Step, Step S43) Eviction Rule Searching Step, Step S44) Cache Page Time Judgment Step, and Step S45) Cache Page Eviction Step.

[0058] Step S41) Cache Page Scanning Step: Create a background scanning program to periodically scan all cache pages.

[0059] Step S42) Cache Page Judgment Step: Judge whether there are cache pages that have not been scanned. If so, execute the next step; if not, end the scanning.

[0060] Step S43) Eviction Rule Searching Step: Based on the longest prefix matching rule, search whether the directory path in the cache page that has not been scanned can match the directory path corresponding to the eviction rule in the updated cache eviction rule. If not, return to the cache page judgment step; if so, define the eviction rule in the updated cache eviction rule that the directory path in the cache page that has not been scanned can match as the matching eviction rule, and execute the next step.

[0061] Step S44) Cache Page Time Judgment Step: Based on the matching eviction rule, judge whether the cache page that has not been scanned has expired. If not, return to the cache page judgment step; if so, execute the next step.

[0062] Step S45) Cache Page Eviction Step: Evict the cache page that has not been scanned and judge whether the eviction is successful. If so, return to the cache page judgment step; if not, record the error information log.

[0063] In this embodiment, to ensure that the system can efficiently manage cached data, a timing scan mechanism is set up. This mechanism periodically scans all cached data according to the time intervals preset by the system (such as every minute, every hour, etc.). In each scan, the current cached data items are traversed, and it is checked whether the survival time of each data item has exceeded the set TTL eviction duration. This timing scan design ensures the real-time performance and effectiveness of the cache, enabling timeout data to be quickly identified and processed.

[0064] During the scan, eviction operations are performed on the timeout data items. Specifically, when it is detected that the survival time of a certain data item has exceeded its TTL eviction duration, the data item is removed from the cache, and the occupied memory resources are released. The execution of the eviction operation triggers the corresponding recycling mechanism to ensure that the memory usage rate of the system is maintained within a reasonable range, avoiding performance degradation caused by the accumulation of timeout data. And the logs of the eviction operations are recorded for subsequent auditing and system performance analysis.

[0065] In the application scenario of this embodiment, for example, Use the TTLadd command to add a TTL rule. For example, the following adds a timeout duration of 1h for the / s3 / test / directory and a timeout duration of 5s for the / s3 / test_folder / directory.

[0066] . / bin / alluxio ttl add --path <path>–time <time> . / bin / alluxio ttl add --path / s3 / test / --time 1h Added alluxioPath= / s3 / test / and time=1h . / bin / alluxio ttl add --path / s3 / test_folder / --time 5s Warning: You are setting TTL policy to 5s. This TTL is too small.Note that expired cache are scanned and evicted every 1h. Please considermaking this TTL larger with `bin / alluxio ttl update` command. Added alluxioPath= / s3 / test_folder / and time=5s An error will be reported when the eviction rule is illegal, such as when the timeout duration is negative.

[0067] . / bin / alluxio ttl add --path / s3 / test_folder / --time -1h Failed to set the TTL policy time with value -1h. You can only setTTL policy time with positive value. Error: error running add --- at / Users / siyuansheng / Desktop / enterprise / cli / src / alluxio.org / cli / env / command.go:126 (BaseJavaCommand.RunWithIO) --- Caused by: exit status 1 Usage: bin / alluxio ttl add [flags] Examples: # Add a TTL policy of 1 hour to the path / s3 / a in Alluxio namespace $ bin / alluxio ttl add --path / s3 / a --time 1h Flags: --attach-debug True to attach debug opts specified by $ALLUXIO_USER_ATTACH_OPTS -h, --help help for add -D, --java-opts strings Alluxio properties to apply, ex. -Dkey=value --path string Alluxio path to set TTL policy on --time string The time to live for this path, eg. 1h, 2d Global Flags: --debug-log True to enable debug logging --skip-license-warnings Skip the automatic warnings before expiration Next, try to update the added rule to a different time interval.

[0068] . / bin / alluxio ttl update --path <path>–time <time> . / bin / alluxio ttl update --path / s3 / test_folder / --time 30min Updated alluxioPath= / s3 / test_folder / and time=30min . / bin / alluxio ttl update --path / s3 / test_folder / --time 5s Warning: You are setting TTL policy to 5s. This TTL is too small.Note that expired cache are scanned and evicted every 1h. Please considermaking this TTL larger with `bin / alluxio ttl update` command. Updated alluxioPath= / s3 / test_folder / and time=5s Use the ttl remove command to delete the current specified ttl rule.

[0069] . / bin / alluxio ttl remove --path <path> . / bin / alluxio ttl remove --path / s3 / test_folder Removed TTL policy for alluxioPath= / s3 / test_folder / Use the ttl list command to view the currently effective ttl rules.

[0070] . / bin / alluxio ttl list Listing all TTL policies The ttl policy table is empty. . / bin / alluxio ttl list Listing all TTL policies / s3 / test.csv / TTL: 1 hour / s3 / test_2.csv / TTL: 30 seconds This application also proposes a storage medium 100, storing computer-readable instructions, when the computer-readable instructions are read by at least one processor 110, enabling at least one processor 120 to execute at least one step in the method for identifying the authenticity of multiple invoice contents.

[0071] This application provides an Alluxio nested cache automatic eviction method and a storage medium. First, adopt the directory-level TTL rule, store the TTL rule in the form of a key-value pair of directory path and TTL eviction duration in etcd, and synchronize it to all Worker nodes through the heartbeat mechanism, simplifying the management of large-scale file metadata; second, through the timed scanning mechanism, the system regularly checks the survival time of cached data and automatically evicts the timeout data to ensure that only valid data is retained in the cache, improving the cache space utilization rate; finally, support dynamic adjustment of the TTL rule, users can flexibly add, update or delete rules through CLI commands, and the system can quickly respond to rule changes to meet the needs of different application scenarios. These technical means significantly reduce the complexity of cache management, reduce the system burden, and at the same time improve the data access performance and system response efficiency, and are especially suitable for big data and distributed computing environments.

[0072] The above has introduced in detail a method for automatically evicting Alluxio nested caches and a storage medium provided by this application. In this article, specific examples are used to elaborate on the principle and implementation manner of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to this application.< / path> < / time> < / path> < / time> < / path>

Claims

1. An Alluxio nested cache automatic eviction method, characterized in that: The steps include: A database establishment step, establishing a distributed key-value database for storing a cache eviction rule set, wherein the cache eviction rule set includes at least one cache eviction rule, and the cache eviction rule is stored in the form of a key-value pair consisting of a directory path and a TTL eviction duration; In the step of issuing the expulsion rule, the user issues different expulsion rule commands through a command line interface (CLI); An eviction rule update step, based on the eviction rule command, periodically interacts each Alluxio Worker node with the distributed key-value database to update the cache eviction rule set; as well as The cache eviction management step periodically checks the data in each cache page in the local file system and clears the timed cache pages, wherein the cache page includes at least one directory path.

2. The Alluxio nested cache automatic eviction method as claimed in claim 1, characterized in that: When the eviction rule command is an add eviction rule command, the eviction rule publishing step includes the following steps: The eviction rule building step builds an add eviction rule; A legal verification step, determining whether the directory path of the added eviction rule is correct and whether the TTL eviction time of the added eviction rule is legal, if so, executing the next step, if not, issuing an abnormal notification; as well as The eviction rule standardization step standardizes the directory path and TTL eviction duration of the eviction rule to be added to form a standardized eviction rule, and stores the standardized eviction rule in the distributed key-value database.

3. The Alluxio nested cache automatic eviction method according to claim 1, wherein: When the eviction rule command is a delete eviction rule command, the eviction rule publishing step includes: A directory path verification step, determining whether the directory path corresponding to the eviction rule requested to be deleted exists, if so, executing the next step, if not, issuing an exception notification; and The eviction rule deletion step deletes the eviction rule requested to be deleted.

4. The Alluxio nested cache automatic eviction method as claimed in claim 1, characterized in that: When the eviction rule command is an update eviction rule command, the eviction rule publishing step includes: An eviction rule verification step, determining whether the eviction rule requested to be updated exists, if so, executing the next step, if not, issuing an exception notification; and The eviction rule updating step configures a new TTL eviction time for the directory path corresponding to the eviction rule requested to be updated, and updates the eviction rule requested to be updated.

5. The Alluxio nested cache automatic eviction method as claimed in claim 1, characterized in that: When the eviction rule command is a view eviction rule command, the directory path and TTL eviction duration of each eviction rule in the cache eviction rule set are displayed in a structured format.

6. The Alluxio nested cache automatic eviction method as claimed in claim 1, characterized in that: The eviction rule updating step comprises the following steps: A command judgment step, wherein each Alluxio Worker node periodically performs heartbeat interaction with the distributed key-value database, and the Alluxio Worker node pulls the cache eviction rule set from the distributed key-value database, and judges whether the eviction rule command is legal based on the cache eviction rule set. If so, the next step is executed; if not, an exception notification is issued; and The command identification step determines whether the eviction rule command is a check eviction rule command. If so, the latest cache eviction rule is checked. If not, the cache eviction rule set is updated according to the eviction rule command to obtain an updated cache eviction rule set.

7. The Alluxio nested cache automatic eviction method as claimed in claim 1, characterized in that: The cache eviction management step includes the following steps: Cache page scanning step, create a background scanner to periodically scan all cache pages; A cache page determination step is to determine whether there are cache pages that have not been scanned. If so, the next step is executed. If not, the scan is terminated. The eviction rule search step is based on the longest prefix matching rule to find out whether the directory path in the unscanned cache page can match the directory path corresponding to the eviction rule in the update cache eviction rule. If not, return to the cache page judgment step. If so, define the eviction rule in the update cache eviction rule that the directory path in the unscanned cache page can match as the matching eviction rule, and execute the next step; A cache page time judgment step, based on the matching eviction rule, judging whether the cache page that has not been scanned is expired, if not, returning to the cache page judgment step, if yes, executing the next step; as well as The cache page eviction step evicts the cache page that has not been scanned and determines whether the eviction is successful. If so, returns to the cache page determination step. If not, records an error information log.

8. A storage medium storing computer-readable instructions, which, when read by at least one processor, causes at least one processor to execute at least one step of the Alluxio nested cache automatic eviction method as described in any one of claims 1 to 7.

Images