Distributed file system and method for deleting its timing snapshot, and electronic device

Abstract

The application discloses a kind of distributed file system and its timing snapshot deletion method, electronic equipment, it is related to storage technical field.Therein, method includes for distributed file system configuration system level reservation rule, for each storage directory configuration and directory level reservation rule corresponding to timing snapshot strategy.Matching is carried out to the attribute information of traversed timing snapshot and directory level reservation rule, system level reservation rule according to the order of priority from high to low.If timing snapshot attribute information and the reservation condition of current matching reservation rule match, send delete request to the metadata service node of belonging, to delete corresponding timing snapshot.If all reservation conditions are not matched, continue to be matched with the reservation rule of next priority.The application can solve the problem that related art cannot meet the efficient data life cycle management needs of large-scale and multi-service scenarios, realize the automatic management of snapshot life cycle, and improve the intelligence, automation and efficiency of distributed file system.

Description

Technical Field

[0001] This invention relates to the field of storage technology, and in particular to a distributed file system and its timed snapshot deletion method, as well as an electronic device. Background Technology

[0002] Distributed file systems rely on distributed file storage technology to achieve data distribution and redundancy, providing the infrastructure to support large-scale data storage and adapting to the needs of automated management in dynamic environments. Related technologies use timed snapshots to allow users to configure flexible policies for different directories to achieve automatic snapshot creation. However, this can lead to wasted storage resources, increased system management complexity, and difficulty in meeting the needs of efficient data lifecycle management in large-scale, multi-business scenarios.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] This invention provides a distributed file system and its timed snapshot deletion method and electronic device, realizing automated lifecycle management of snapshots and improving the intelligence, automation and efficiency of the distributed file system.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a method for timed snapshot deletion in a distributed file system, comprising: Configure directory-level retention rules corresponding to at least one timed snapshot strategy for different storage directories of the distributed file system, and configure system-level retention rules for the distributed file system. The directory-level retention rules and the system-level retention rules each include at least one of time-based retention conditions and number-based retention conditions. During the traversal of each timed snapshot, the attribute information of the current timed snapshot is matched with the directory-level retention rule and the system-level retention rule in descending order of priority; wherein, the directory-level retention rule has a higher priority than the system-level retention rule. If the attribute information of the current timed snapshot matches the retention conditions in the currently matched retention rule, a request to delete the timed snapshot is sent to the target metadata service node to which the target storage directory to which the current timed snapshot belongs, so that the target metadata service node performs the corresponding snapshot deletion operation; if the attribute information of the current timed snapshot does not match any retention conditions in the currently matched retention rule, then it continues to match with the retention rule of the next priority.

[0006] Another aspect of the present invention provides a distributed file system, including a metadata service management node and multiple metadata service nodes connected thereto; The metadata service management node is configured to execute the steps of the timed snapshot deletion method of the above distributed file system. Whenever a timed snapshot deletion request is sent to the metadata service node, the index is traversed to point to the next timed snapshot and the corresponding matching operation is performed. While the metadata service node is performing the snapshot deletion operation, the matching and deletion requests of other timed snapshots are processed in parallel without waiting for the metadata service node to return the response result. Upon receiving a request to delete a scheduled snapshot from the metadata service management node, each metadata service node executes the corresponding scheduled snapshot deletion operation, responds to the metadata service management node with the snapshot deletion result, and simultaneously synchronizes the snapshot deletion result to other metadata service nodes in the distributed file system.

[0007] The present invention also provides an electronic device, including a memory and a processor, wherein the processor is used to implement the steps of the timed snapshot deletion method of the distributed file system described above when executing a computer program stored in the memory.

[0008] The advantages of the technical solution provided by this invention are that by configuring directory-level retention rules corresponding to the timed snapshot strategy for different storage directories in the distributed file system, and configuring system-level retention rules for the entire file system, and setting the matching priority of directory-level retention rules to be higher than that of system-level retention rules, when traversing each timed snapshot, fine-grained matching can be performed first at the directory level according to the retention conditions corresponding to the snapshot's own strategy type. Only when the directory-level rules cannot match will the unified system-level rules be used for fallback processing. This layered matching method from precise to generalized achieves differentiated and fine-grained management of the snapshot lifecycle under different business scenarios and different directories through directory-level rules, avoiding the one-size-fits-all problem caused by the unified strategy. On the other hand, the system-level rules provide a unified expiration cleanup guarantee for snapshots that are not configured with directory-level rules or cannot be covered by directory-level rules, preventing the infinite accumulation of snapshots. Based on this, when snapshot attribute information matches any priority retention condition, the designated metadata service node sends a deletion request to the metadata service node to which the target storage directory belongs. This achieves collaborative snapshot deletion operations in a distributed environment, ensuring the uniformity of matching decisions and avoiding performance bottlenecks caused by a single node performing deletion operations. This enables refined and automated snapshot lifecycle management, balancing data protection and system resource utilization efficiency. While reducing storage resource consumption and alleviating metadata service pressure, it significantly improves the operational efficiency and system stability of the distributed file system in multi-business scenarios. Furthermore, this invention also provides corresponding electronic devices and a distributed file system for the timed snapshot deletion method of the distributed file system, further enhancing the practicality of the method. The electronic devices and distributed file system offer corresponding advantages. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the present invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a timed snapshot deletion method for a distributed file system provided by the present invention; Figure 2 A flowchart illustrating another timed snapshot deletion method for a distributed file system provided by the present invention; Figure 3 A structural framework diagram of an exemplary embodiment of the timed snapshot deletion device for the distributed file system provided by the present invention; Figure 4A structural diagram of an exemplary embodiment of the distributed file system provided by the present invention; Figure 5 The flowchart of the timed snapshot deletion method for the distributed file system provided by the present invention is shown in an exemplary application scenario. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. In this specification and the aforementioned drawings, the terms "first," "second," "third," "fourth," etc., are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. The term "exemplary" means "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0012] Distributed file storage is a cluster of servers with multiple storage nodes. Files are stored in chunks, with objects as the basic unit. It supports storing a single piece of data across multiple nodes, and each node can obtain the complete data through inter-node communication. When a node fails, complete data recovery can be performed according to configured policies. It features high availability, high performance, and high scalability. File semantics are provided by each node's MDS (Metadata Service), used for various metadata access operations, balancing business load. In distributed storage cluster environments, with the growth of data scale and the popularization of cloud computing and big data applications, distributed file systems bear increasingly important data access responsibilities. To ensure data recoverability in the face of accidental operations, software defects, or external attacks, snapshot technology has been introduced as a core data protection mechanism. Compared to traditional backup solutions, snapshots are characterized by rapid creation, efficient recovery, and relatively low storage costs. However, relying on manually triggered snapshot creation methods can no longer meet the automation needs of highly dynamic business scenarios. Therefore, timed snapshot technology, which supports automatic execution on a time cycle, has emerged as a key evolutionary direction for improving data management efficiency.

[0013] While scheduled snapshot technology solves the problem of automated creation, the number of snapshots accumulates over time, and a lack of a reasonable deletion strategy leads to storage space being occupied by a large amount of redundant data, while also increasing the retrieval and maintenance pressure on metadata services. Secondly, although snapshots share some data blocks, long-term redundant snapshots still significantly increase the overall load and operating costs of the storage cluster. Furthermore, in complex scenarios with multiple directories, multiple policies, and multiple tenants, it is difficult to accurately track and manually clean up the lifecycle of a large number of snapshots, increasing the complexity of operation and maintenance and the risk of errors. Therefore, this invention supports user-defined snapshot retention rules (interval policy, daily policy, weekly policy, monthly policy, etc.) for each directory with different scheduled policies. It supports retention rules by time and by number, and supports simultaneous configuration of retention rules for multiple policies, increasing the flexibility of scheduled snapshot management. Simultaneously, the distributed file system as a whole supports configuring a unified retention rule. During automatic scheduled snapshot management, it first checks whether the retention rule for the scheduled snapshot policy type is met. If retention is required, it further checks the system's scheduled snapshot expiration time. In this way, while enjoying the convenience of scheduled snapshots, it achieves automated lifecycle management of snapshots, balancing data protection and system resource utilization efficiency, and improving the intelligence, automation, and efficiency of the distributed storage system. After introducing the technical solution of the present invention, various non-limiting embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Please refer to [link to previous text] first. Figure 1 According to the present invention, a method for deleting timed snapshots in a distributed file system can be implemented as a computer program product, installed and run in the metadata server of the distributed file system, for implementing timed snapshot deletion processing. During the operation of the distributed file system, a metadata service node is specified, such as the node with sequence number zero in the metadata service cluster, to execute the following timed snapshot deletion method. In some embodiments of the method, the method includes the following steps: S101: Configure directory-level retention rules corresponding to at least one timed snapshot strategy for different storage directories of the distributed file system, and configure system-level retention rules for the distributed file system.

[0014] The distributed file system consists of a cluster of multiple storage node servers. Files are stored in blocks, with objects as the basic unit. The metadata service cluster is responsible for maintaining the file system's directory tree and metadata information. Directory-level retention rules are snapshot retention rules configured individually for different storage directories within the distributed file system, corresponding to the scheduled snapshot policy. These are fine-grained snapshot management rules for specific storage directories and can be flexibly set according to the business needs of different directories. System-level retention rules are snapshot retention rules uniformly configured for the entire distributed file system. They serve as a fallback rule to the directory-level retention rules, providing unified management for snapshots without configured directory-level rules or those not covered by them. Scheduled snapshot policies are the time-based strategies that trigger the distributed file system to automatically create snapshots. These include interval-based snapshot policies and periodic snapshot policies (daily, weekly, monthly, etc.).

[0015] In this step, to achieve automated lifecycle management of scheduled snapshots, system administrators can configure directory-level retention rules corresponding to specific scheduled snapshot policies for different storage directories based on business needs. These directory-level retention rules must include at least one of two conditions: retention by time and retention by number. For example, for directories storing core business data, a daily policy can be configured with directory-level retention rules, where the time retention condition is set to retain for 7 days and the retention by number condition is set to retain a maximum of 10 snapshots. For directories storing temporary data, an interval policy can be configured with directory-level retention rules, where the time retention condition is set to retain for 1 day. Simultaneously, system administrators also configure system-level retention rules for the entire distributed file system. These system-level retention rules must include at least one of two conditions: retention by time and retention by number. For example, a unified snapshot expiration time can be set to 30 days, a maximum number of snapshots per directory can be set to 100, and a global maximum number of snapshots can be set to 10,000.

[0016] S102: During the traversal of each timed snapshot, the attribute information of the current timed snapshot is matched with the directory-level retention rules and the system-level retention rules in descending order of priority.

[0017] The attribute information refers to the characteristic information of the timed snapshot, which may include the snapshot creation time, the storage directory identifier, and the corresponding timed snapshot policy type. During the operation of the distributed file system, all timed snapshots are traversed at a preset time period (e.g., every 5 seconds). The current timed snapshot refers to the timed snapshot traversed at the current moment or a specified moment during the traversal. If the snapshot type being traversed is a manually created snapshot type, all matching steps are skipped, and no deletion operation is performed. If the snapshot type being traversed is not a manually created snapshot type, since directory-level retention rules have higher priority than system-level retention rules, the system first attempts to judge using more refined directory-level rules. That is, the attribute information of the current timed snapshot (including the snapshot creation time, the storage directory identifier, the timed snapshot policy type, etc.) is first matched with the directory-level retention rules, and then matched with the system-level retention rules.

[0018] S103: If the attribute information of the current timed snapshot matches the retention conditions in the current matching retention rule, a request to delete the timed snapshot is sent to the target metadata service node to which the target storage directory of the current timed snapshot belongs, so that the target metadata service node can perform the corresponding snapshot deletion operation; if the attribute information of the current timed snapshot does not match any retention conditions in the current matching retention rule, then it continues to match with the retention rule of the next priority.

[0019] In this system, the target metadata service node is responsible for maintaining the current timed snapshot, and the target storage directory is the storage directory to which the current timed snapshot belongs. If the attribute information of the current timed snapshot matches the retention conditions in the directory-level retention rules (such as retention conditions by time or retention conditions by number), it is determined that the snapshot needs to be deleted. At this time, the specified metadata service node sends a delete timed snapshot request to the target metadata service node to which the target storage directory to which the snapshot belongs, and the target metadata service node performs the actual snapshot deletion operation. Conversely, if the attribute information of the current timed snapshot does not match any retention conditions in the directory-level retention rules, the system continues to match the snapshot's attribute information with the next higher priority system-level retention rules to determine whether deletion is necessary.

[0020] In the technical solution provided in this embodiment, by configuring directory-level retention rules corresponding to the timed snapshot strategy for different storage directories in the distributed file system, and configuring system-level retention rules for the entire file system, and setting the matching priority of directory-level retention rules to be higher than that of system-level retention rules, when traversing each timed snapshot, fine-grained matching can be performed first at the directory level according to the retention conditions corresponding to the snapshot's own strategy type. Only when the directory-level rules cannot match will the system-level unified rules be used for fallback processing. This layered matching method from precise to generalized achieves differentiated and fine-grained management of the snapshot lifecycle under different business scenarios and different directories through directory-level rules, avoiding the one-size-fits-all problem caused by unified strategies. On the other hand, system-level rules provide a unified expiration cleanup guarantee for snapshots that are not configured with directory-level rules or cannot be covered by directory-level rules, preventing snapshots from accumulating indefinitely. Based on this, when the snapshot attribute information matches any priority retention condition, the designated metadata service node sends a deletion request to the metadata service node to which the target storage directory belongs. This realizes the division of labor and collaboration in snapshot deletion operations in a distributed environment, ensuring the uniformity of matching decisions and avoiding the performance bottleneck caused by a single node performing deletion operations. This achieves refined and automated snapshot lifecycle management, balancing data protection and system resource utilization efficiency. While reducing storage resource consumption and alleviating the pressure on metadata services, it significantly improves the operation and maintenance efficiency and system stability of the distributed file system in multiple business scenarios.

[0021] Based on the above embodiments, this embodiment refines the matching process of retention rules to achieve precise positioning of directory-level retention rules, which may include the following: Based on the storage directory identifier and timed snapshot policy type of the current timed snapshot, the unique corresponding directory-level retention rule is located from the directory-level retention rules configured for different storage directories and corresponding to different timed snapshot policies. The timed snapshot policy type of the current timed snapshot is an interval timed snapshot policy. Based on the storage directory identifier, the interval policy directory-level retention rule corresponding to the storage directory identifier is located from the directory-level retention rules configured for different storage directories and corresponding to the interval timed snapshot policy. The snapshot policy type of the current timed snapshot is a periodic timed snapshot policy. Based on the storage directory identifier, the directory-level retention rule corresponding to the storage directory identifier and matching the periodic timed snapshot policy is located from the directory-level retention rules configured for different storage directories and corresponding to the periodic timed snapshot policy.

[0022] The storage directory identifier is information used in the distributed file system to uniquely identify each storage directory, allowing for precise location of the storage directory to which a snapshot belongs. Interval-based snapshot policies automatically trigger snapshot creation at preset time intervals, starting from a specified time, such as creating a snapshot every 2 hours. Periodic snapshot policies automatically trigger snapshot creation at fixed time periods, including daily (e.g., creating at 2 AM every day), weekly (e.g., creating at 3 AM every Sunday), and monthly (e.g., creating at 4 AM on the 1st of every month) policies. The period type must include at least one of the following: daily, weekly, or monthly.

[0023] In this embodiment, if the current timed snapshot's timed snapshot policy type is an interval timed snapshot policy, the designated metadata service node locates the interval policy directory-level retention rule corresponding to the storage directory identifier from the directory-level retention rules configured for different storage directories and corresponding to the interval timed snapshot policy. For example, for a snapshot generated by an interval policy (created every 6 hours) under the directory / data / business_a, the system locates the interval policy directory-level retention rule corresponding to the directory from the configuration table based on the directory identifier / data / business_a and the policy type interval policy. If the current timed snapshot's snapshot policy type is a periodic timed snapshot policy, the designated metadata service node locates the directory-level retention rule corresponding to the storage directory identifier and matching the periodic timed snapshot policy from the directory-level retention rules configured for different storage directories and corresponding to the periodic timed snapshot policy. For example, for a snapshot generated by a daily policy (created at 2 AM every day) under the directory / data / business_b, the system locates the daily policy directory-level retention rule corresponding to the directory based on the directory identifier / data / business_b and the policy type daily policy.

[0024] As shown above, this embodiment achieves unique and precise location of directory-level retention rules, solving the problem of chaotic retention rule matching in multi-directory and multi-policy scenarios, and improving the accuracy of snapshot retention rule matching. It categorizes and matches different types of timed snapshot policies, adapting to the flexible and diverse snapshot creation strategies in distributed file systems, further enhancing the method's versatility and adaptability. Precise rule location reduces invalid rule matching processes, improves the efficiency of snapshot traversal and matching, and reduces the processing pressure on metadata service nodes.

[0025] Based on the above embodiments, this embodiment refines the matching process of directory-level retention rules to achieve precise positioning of directory-level retention rules, which may include the following: If the directory-level retention rule includes a time-based retention condition, a successful match is determined if the sum of the creation time of the current timed snapshot and the retention time set by the time-based retention condition is less than or equal to the current time. If the directory-level retention rule includes a number-based retention condition, the number of all timed snapshots with the same timed policy type under the target storage directory is counted. A successful match is determined if the count of related timed snapshots exceeds the maximum number set by the number-based retention condition.

[0026] Among them, the retention condition based on time refers to the retention condition set in the retention rule based on the snapshot creation time. It determines whether a snapshot meets the deletion criteria by calculating the snapshot creation time and the preset retention time. The retention condition based on the number of snapshots with the same policy in the retention rule refers to the retention condition set in the retention rule. It determines whether a snapshot meets the deletion criteria by counting the total number of snapshots of the same timed snapshot policy type in the specified storage directory. The maximum number is the maximum number of snapshots of the same timed snapshot policy type in the specified storage directory that can be retained as set in the retention condition based on the number.

[0027] In this embodiment, during the check for time-based retention conditions, if the current directory-level retention rule includes a time-based retention condition, the system adds the creation time of the current timed snapshot to the retention time set by the time-based retention condition to obtain the snapshot's expiration time. This expiration time is then compared to the current time. If the expiration time is less than or equal to the current time, it indicates that the snapshot has expired, the system determines a successful match, and triggers the deletion process. For example, if a snapshot was created on January 1, 2024, at 00:00, and its corresponding time-based retention condition sets a retention time of 7 days, then its expiration time is January 8, 2024, at 00:00. If the current time is January 9, 2024, at 00:00, then the snapshot has expired, and the match is successful. During the check for quantity-based retention conditions, if the current directory-level retention rule includes a quantity-based retention condition, the system counts all timed snapshots with the same timed policy type in the target storage directory to which the snapshot belongs, and compares this count with the maximum number set by the quantity-based retention condition. If the number of snapshots obtained exceeds the set limit, the system determines that a match has been successfully made and triggers the deletion process. For example, if there are 15 timed snapshots generated by the interval strategy in a certain directory, and the interval strategy for that directory sets the maximum number of snapshots to be retained to 10, then the snapshots exceeding this limit will be determined as a successful match and will be deleted. Usually, the earliest created snapshots are deleted in order of their creation time from earliest to latest.

[0028] As shown above, this embodiment clearly defines the matching criteria for directory-level retention rules. Based on the snapshot's creation time and the number of snapshots of the same type, it accurately determines whether a snapshot has expired or exceeded its quantity limit, improving the accuracy of snapshot deletion. It supports the individual or combined use of retention conditions based on time or quantity, further enhancing the flexibility of directory-level retention rules. These rules can be flexibly configured according to the needs of different business scenarios, adapting to more diverse snapshot management requirements. The entire judgment logic is simple and efficient, completing the judgment based on the statistics of snapshot creation time and quantity, reducing the computational pressure on metadata service nodes and ensuring the operational efficiency of the distributed file system.

[0029] Based on the above embodiments, this embodiment refines the matching process of system-level retention rules, effectively manages snapshots that do not match directory-level rules, and achieves full-dimensional management of the snapshot lifecycle, avoiding the problems of snapshot accumulation and storage resource waste. This may include the following: If the system-level retention rule includes a unified snapshot expiration time, a successful match is determined if the sum of the creation time of the current scheduled snapshot and the unified snapshot expiration time is less than or equal to the current time. If the system-level retention rule includes a maximum number of snapshots for a directory, a successful match is determined if the total number of snapshots under the target storage directory to which the current scheduled snapshot belongs is counted and the total number of local snapshots exceeds the maximum number of snapshots for the directory. If the system-level retention rule includes a global maximum number of snapshots, a successful match is determined if the total number of snapshots under the distributed file system exceeds the global maximum number of snapshots.

[0030] The unified snapshot expiration time is a standardized snapshot retention period set for the entire distributed file system within the system-level retention rules; it serves as a universal expiration time standard for all snapshots. The maximum number of snapshots per directory is the maximum number of snapshots that can be retained under a single storage directory, as set in the system-level retention rules. The global maximum number of snapshots is the total number of snapshots that can be retained across the entire distributed file system, as set in the system-level retention rules. The total number of local snapshots is the sum of all snapshots under a single storage directory, including snapshots created using different timed snapshot strategy types. The total number of global snapshots is the sum of snapshots across all storage directories in the entire distributed file system.

[0031] In this implementation, when the specified metadata service node matches the attribute information of the current scheduled snapshot with the system-level retention rules, the system sequentially performs checks on the unified snapshot expiration time, the maximum number of snapshots in the directory, and the global maximum number of snapshots. In the unified snapshot expiration time check, if the system-level retention rules include a unified snapshot expiration time, the system adds the creation time of the current scheduled snapshot to the unified snapshot expiration time to obtain the system-level expiration time of the snapshot, and compares this expiration time with the current time. If the expiration time is less than or equal to the current time, it indicates that the snapshot has exceeded the system-defined retention period, the system determines a successful match, and triggers the deletion process. In the maximum number of snapshots in the directory check, if the system-level retention rules include a maximum number of snapshots in the directory, the system counts the total number of snapshots (including manual snapshots and scheduled snapshots) under the target storage directory to which the current scheduled snapshot belongs, and compares the counted number of local snapshots with the maximum number of snapshots in the directory. If the counted number of local snapshots exceeds the maximum number of snapshots in the directory, the system determines a successful match, and triggers the deletion process. During the check of the global maximum snapshot count, if the system-level retention rule includes a global maximum snapshot count, the system counts the total number of snapshots across the entire distributed file system and compares this total global snapshot count with the global maximum snapshot count. If the total number of global snapshots exceeds the global maximum snapshot count, the system determines that a match has been successfully established and triggers the deletion process.

[0032] As can be seen from the above, this embodiment can determine the deletion of timed snapshots based on multiple dimensions such as unified snapshot expiration time, maximum number of snapshots in the directory, and maximum number of global snapshots, to prevent snapshots from accumulating indefinitely and ensure that the snapshot scale of the entire file system is within a controllable range.

[0033] Based on the above embodiments, in order to ensure the accuracy and robustness of the matching process, this implementation adds a series of pre-check operations before matching the attribute information of the current timed snapshot with the directory-level retention rules and the system-level retention rules. These operations may include the following: If there are incomplete snapshot creation operations in the target storage directory of the current timed snapshot, the matching operation for the current timed snapshot is paused. If there are no incomplete snapshot creation operations in the target storage directory of the current timed snapshot, and if the directory-level retention rules of the target storage directory of the current timed snapshot have been completely deleted, the matching operation with the directory-level retention rules is skipped, and the system-level retention rules are matched directly. If the directory-level retention rules of the target storage directory of the current timed snapshot have not been completely deleted, and there is no retention rule in the directory-level retention rules of the target storage directory of the current timed snapshot that corresponds to the timed snapshot policy type of the current timed snapshot, the matching operation with the directory-level retention rules is skipped, and the system-level retention rules are matched directly. If there is a retention rule in the directory-level retention rules of the target storage directory of the current timed snapshot that corresponds to the timed snapshot policy type of the current timed snapshot, and the current timed snapshot does not record a corresponding timed snapshot policy type, the matching operation with the directory-level retention rules is skipped, and the system-level retention rules are matched directly.

[0034] The snapshot creation operation is the process by which the distributed file system automatically creates snapshots according to a scheduled snapshot strategy, including steps such as snapshot metadata generation and data block association. An incomplete snapshot creation operation is one that is in progress but has not yet completed all steps; at this stage, the snapshot's attribute information has not been fully generated or synchronized.

[0035] In this embodiment, firstly, the system checks whether there are any incomplete snapshot creation operations in the target storage directory to which the current timed snapshot belongs. If there are incomplete snapshot creation operations, the system pauses the matching operation for the current timed snapshot under that storage directory and waits for the snapshot creation operation to be completed before continuing processing. This ensures the accuracy of the count based on the retention rule and avoids inaccurate statistics due to incomplete creation. Secondly, after confirming that there are no incomplete snapshot creation operations in the target storage directory, the system further checks whether the directory-level retention rule for that directory has been completely deleted. If it has been completely deleted, the system skips the matching operation with the directory-level retention rule and directly matches the snapshot's attribute information with the system-level retention rule. Thirdly, if the directory-level retention rule has not been completely deleted, the system checks whether there is a retention rule in the directory-level retention rule that corresponds to the timed snapshot policy type of the current timed snapshot. If not (for example, the snapshot is created by an interval policy, but the administrator subsequently changes the interval policy retention rule for that directory to a daily policy retention rule), the system skips the matching operation with the directory-level retention rule and directly matches it with the system-level retention rule. Finally, if a directory-level retention rule exists that corresponds to the current timed snapshot's policy type, the system checks whether the current timed snapshot records a corresponding timed snapshot policy type. If the snapshot does not record a policy type (for example, during a system upgrade, a snapshot created in an older version does not record policy type information), the system skips the matching operation with the directory-level retention rule and directly matches it with the system-level retention rule.

[0036] As shown above, this embodiment avoids performing matching and deletion operations on snapshots in the process of creation, preventing incomplete snapshots from being accidentally deleted, ensuring the data integrity of the distributed file system, and guaranteeing the security of snapshot data. By performing pre-checks on the configuration status of directory-level retention rules and the status of snapshot policy type records, invalid rule matching and matching errors are avoided, making the snapshot deletion determination more consistent with the actual configuration and improving the accuracy of rule matching. By optimizing the matching process, directory-level rule matching is skipped directly in cases of abnormal rule configuration, reducing invalid calculations and processing, and improving the overall efficiency of snapshot traversal and matching.

[0037] Based on the above embodiments, this embodiment, in order to ensure the orderly execution of the deletion operation, also adds an anti-duplicate processing method during the snapshot deletion operation, which may include the following: Set a directory deletion flag for the target storage directory to which the current timed snapshot belongs. The directory deletion flag is used to prevent the issuance of a new snapshot deletion request to the target storage directory while there are incomplete snapshot deletion operations in the target storage directory. The directory deletion flag is cleared after receiving the snapshot deletion result response from the target metadata service node.

[0038] The snapshot deletion result response is the operation result information returned by the metadata service node to the node that initiated the deletion request after the snapshot deletion operation is completed. It may include statuses such as successful deletion or deletion failure. When a specified metadata service node decides to initiate a deletion operation on the current timed snapshot, it first sets a directory deletion flag for the target storage directory before sending a deletion request to the target metadata service node to which the snapshot belongs. The purpose of this directory deletion flag is to prevent the issuance of new snapshot deletion requests to the target storage directory while there are incomplete snapshot deletion operations. This avoids data races and state inconsistencies that may be caused by the concurrent deletion of multiple snapshots in the same directory. When the specified metadata service node receives the snapshot deletion result response from the target metadata service node, it indicates that the snapshot deletion operation has been completed. At this time, the system clears the directory deletion flag for the directory, releases the lock on the deletion operation for the directory, and allows subsequent deletion requests to be issued for other snapshots in the directory.

[0039] As can be seen from the above, this embodiment can ensure that only one snapshot deletion operation is performed in the same directory at any given time in a distributed environment by introducing directory-level deletion markers. This ensures the serial execution of the deletion process, avoids concurrent deletion operations on multiple snapshots of the same directory, prevents data races and state inconsistencies, and maintains the consistency of the system state.

[0040] Based on the above embodiments, in order to avoid excessive load on the storage system caused by deletion operations, this embodiment controls the number of deletion requests initiated during a single traversal through a flow control counter, which may include the following: During the process of traversing all timed snapshots, a flow control counter is set. The initial value of the flow control counter is zero. Whenever a request to delete a timed snapshot is sent to the metadata service node, the count value of the flow control counter is incremented by one. After each increment of the flow control counter, the incremented count value is compared with the preset flow control threshold. When the incremented count value reaches the flow control threshold, the current traversal position is recorded and the current traversal is terminated. When the next preset time period arrives, the traversal operation continues from the recorded traversal position.

[0041] The flow control counter is used to count the number of snapshot deletion requests sent, and its value controls the scale of deletion requests sent. The flow control threshold is a preset maximum number of snapshot deletion requests allowed to be sent in a single traversal, and it serves as the judgment threshold for the flow control counter. The preset time period is a fixed time interval for the distributed file system to perform snapshot traversal and matching operations, such as once every 5 seconds or every 10 seconds. The traversal position is the current snapshot node position during the timed snapshot traversal; recording this position allows subsequent traversals to continue running.

[0042] In this embodiment, during the process of traversing all timed snapshots at a designated metadata service node, the system sets a flow control counter, which is initially set to zero. Whenever the system sends a request to delete a timed snapshot to a metadata service node, the flow control counter is incremented by one. After each increment, the system compares the incremented count with a preset flow control threshold (e.g., a maximum of 100 snapshots can be deleted per traversal configured by the administrator). If the incremented count has not yet reached the flow control threshold, the system continues traversing the next timed snapshot and matching the deletion conditions. If the incremented count reaches the flow control threshold, indicating that the number of deletion requests sent in this traversal has reached its limit, the system records the current snapshot position (e.g., records the index number or unique identifier of the current snapshot) and then terminates the current traversal operation. The system waits for the next preset time period (e.g., the next 5 seconds) to arrive, and then resumes the traversal operation from the recorded position to process the remaining snapshots.

[0043] As shown above, this embodiment controls the number of deletion requests per traversal through flow control counters and flow control thresholds. This avoids a sudden increase in system load caused by issuing a large number of deletion requests at once, and prevents excessive load impact on the storage system caused by deleting a large number of snapshots in a short period of time, thus affecting normal business read and write operations and ensuring the normal operation of the distributed file system. By recording the traversal position, subsequent traversals can be resumed, allowing the cleanup of a large number of snapshots to be deleted to be carried out in batches and in an orderly manner. This balances the resource consumption of snapshot cleanup and normal system operation, protecting system stability while ensuring that all snapshots are eventually processed. The flow control threshold can be flexibly adjusted according to the operating load of the distributed file system. For example, the threshold can be increased when the system is under low load to speed up snapshot cleanup, while the threshold can be decreased when the system is under high load to prioritize normal business operations.

[0044] Furthermore, this embodiment adds dynamic flow control based on storage pool load on the basis of global flow control to achieve fine-grained flow control management, which may include the following: During the traversal of all timed snapshots, the current load status of each storage pool in the distributed file system is obtained. Based on the current load status of each storage pool, the flow control threshold corresponding to each storage pool is dynamically adjusted. Specifically, for the first storage pool whose current load status exceeds the preset high load threshold, the flow control threshold value corresponding to the first storage pool is decreased, and for the second storage pool whose current load status is lower than the preset low load threshold, the flow control threshold value corresponding to the second storage pool is increased. Before sending a request to delete a timed snapshot to the target metadata service node, the target storage pool corresponding to the target storage directory to which the current timed snapshot belongs is determined, and the current flow control threshold of the target storage pool is obtained. When the cumulative number of requests to delete timed snapshots sent to the target storage pool during this traversal reaches the current flow control threshold of the target storage pool, the sending of requests to delete timed snapshots to the target storage pool is stopped, and the processing of timed snapshots corresponding to other storage pools continues.

[0045] In a distributed file system, a storage pool is a resource pool used to store data and snapshots. A distributed file system contains multiple independent storage pools, with different storage directories corresponding to different storage pools. The current load status is the real-time operating load of the storage pool, including one or more of the following: the storage pool's input / output queue depth, the average CPU utilization of the storage pool's data nodes, and the average memory utilization of the storage pool's data nodes. The high load threshold is a preset upper limit indicator for the storage pool's load; when the current load status of the storage pool exceeds this threshold, it is considered a high load state. The low load threshold is a preset lower limit indicator for the storage pool's load; when the current load status of the storage pool is below this threshold, it is considered a low load state. The target storage pool is the storage pool corresponding to the target storage directory to which the current timed snapshot belongs; the snapshot's physical data and metadata are stored in this storage pool.

[0046] During the dynamic adjustment of flow control thresholds for each storage pool based on its current load status, for the first storage pool whose current load exceeds a preset high load threshold (e.g., I / O queue depth exceeds 100, CPU utilization exceeds 80%), the system decreases the flow control threshold value for that storage pool to reduce deletion speed and prevent deletion operations from further exacerbating the load pressure on the storage pool. For the second storage pool whose current load is below a preset low load threshold (e.g., I / O queue depth is below 10, CPU utilization is below 30%), the system increases the flow control threshold value for that storage pool to increase deletion speed and fully utilize idle system resources to accelerate snapshot cleanup. Before sending a request to delete a scheduled snapshot to the target metadata service node, the system determines the target storage pool corresponding to the target storage directory to which the current scheduled snapshot belongs and obtains the current flow control threshold of that target storage pool (this threshold has been dynamically adjusted based on the real-time load status of the storage pool). When the system accumulates enough deletion snapshot requests sent to the target storage pool during this traversal to reach the target storage pool's current flow control threshold, it stops sending deletion snapshot requests to that target storage pool, but continues to process snapshots for other storage pools to ensure that deletion operations between different storage pools do not affect each other. If the threshold is not reached, deletion requests are sent normally.

[0047] As shown above, this embodiment considers the load differences of each storage pool and adopts different flow control standards for storage pools with different load states, solving the problem of a one-size-fits-all approach to global flow control and improving resource utilization efficiency. For high-load storage pools, deletion requests are reduced to avoid further load increases and prevent storage pool failures due to excessive load. For low-load storage pools, cleanup speed is accelerated, improving the overall efficiency of snapshot cleanup and ensuring the operational stability of the storage pools. Real-time acquisition of storage pool load status and dynamic adjustment of flow control thresholds ensure that the flow control strategy aligns with the real-time operation of the distributed file system, adapting to complex and dynamic system operating environments.

[0048] Finally, this embodiment also provides an exemplary implementation method in which the metadata service management node MDS0, designated by the distributed file system with the number 0, performs automatic deletion of timed snapshots, such as... Figure 2 As shown, the implementation process includes: A1: MDS0 periodically iterates through timed snapshots.

[0049] A2: If the retention rules for the directory of the timed snapshot do not exist, then execute step A8, which is the file system timed snapshot expiration time and retention number rule check.

[0050] A3: If the policy type for the scheduled snapshot is unknown, proceed to step A8.

[0051] A4: If the retention rules for this timed snapshot directory do not exist for this snapshot policy type, then execute A8.

[0052] A5: If the retention rules for the directory of the timed snapshot exist, then the rules based on time and number will be executed in sequence, followed by A6 and A7.

[0053] A6: Check by time. Add the creation time of the timed snapshot to the retention time and compare it with the current time. If it is greater than the current time, it will not expire and continue to check A7. Otherwise, if it is less than or equal to the current time, it will expire and A9 will be executed to delete it.

[0054] A7: Check by number, obtain the statistics of the number of timed snapshots of different policy types in this directory. If the number of policy types exceeds the retention limit, delete in A9; otherwise, execute A8.

[0055] A8: Performs file system timed snapshot expiration time and retention count checks, similar to A6 and A7. First, it checks if the snapshot expiration time has expired. It adds the creation time of the timed snapshot to the file system snapshot expiration time and compares it with the current time. If it is less than or equal to the current time, it is expired and A9 is executed to delete it. Otherwise, it checks if the number of snapshots for the directory exceeds the limit. If it does, A9 is executed to delete it. Otherwise, it checks again if the number of snapshots for the file system exceeds the limit. If it does, A9 is executed to delete it.

[0056] A9: MDS0 constructs a deletion request for a timed snapshot, sets a directory flag to indicate that the directory is currently undergoing snapshot deletion, and does not allow further deletion requests during this process. The request is sent to the MDS to which the timed snapshot directory belongs, and the process continues to the next snapshot check. The cumulative number of deletion requests is accumulated, and once the flow control limit is reached, the process stops and waits for the next timed cycle to start again.

[0057] A10: After the directory owner MDS receives the data, it performs a snapshot deletion operation and responds to MDS0. Upon receiving the response, MDS0 clears the directory marker.

[0058] As can be seen from the above, this embodiment significantly improves storage resource utilization, reduces operating costs, automatically deletes unnecessary snapshots, avoids storage space waste caused by snapshot accumulation, effectively controls the total number of snapshots and metadata scale, reduces the pressure on distributed metadata services, releases data blocks occupied by snapshots (especially exclusive blocks), and improves the overall resource utilization of the storage cluster; it simplifies operation and maintenance management, enhances the automation and intelligence level of the system, eliminates the need for administrators to manually clean up snapshots periodically, reduces the risk of human error, supports flexible configuration of various retention policies (such as by time, by number, etc.), adapts to complex business scenarios, and the system can automatically perform deletion operations according to the policy without manual intervention, achieving truly unattended snapshot management; the automatic deletion policy can combine data classification and business importance to set different retention and deletion rules for different levels of data, meeting the compliance requirements of highly regulated industries such as finance, healthcare, and government.

[0059] It should be noted that there is no strict order of execution between the steps in this invention. As long as they conform to the logical order, these steps can be executed simultaneously or in a certain preset order. Figures 1-2 This is just an illustrative example and does not mean that this is the only possible execution order.

[0060] This invention also provides a corresponding apparatus for the timed snapshot deletion method of a distributed file system, further enhancing the practicality of the method. The apparatus can be described from both a functional module perspective and a hardware perspective. The timed snapshot deletion apparatus for a distributed file system provided by this invention is described below. This apparatus is used to implement the timed snapshot deletion method for a distributed file system provided by this invention. In this embodiment, the timed snapshot deletion apparatus for a distributed file system may include or be divided into one or more program modules. These one or more program modules are stored in a storage medium and executed by one or more processors to complete the timed snapshot deletion method for a distributed file system disclosed in Embodiment 1. The program module referred to in this embodiment refers to a series of computer program instruction segments capable of performing specific functions, which are more suitable than the program itself for describing the execution process of the timed snapshot deletion apparatus for a distributed file system in the storage medium. The following description will specifically introduce the functions of each program module in this embodiment. The timed snapshot deletion apparatus for a distributed file system described below can be referred to in correspondence with the timed snapshot deletion method for a distributed file system described above.

[0061] From the perspective of functional modules, see Figure 3 , Figure 3 This is a structural diagram of the timed snapshot deletion device for the distributed file system provided in this embodiment, in one specific implementation. The device may include: The rule configuration module 301 is used to configure directory-level retention rules corresponding to at least one timed snapshot strategy for different storage directories of the distributed file system, and to configure system-level retention rules for the distributed file system. Both the directory-level retention rules and the system-level retention rules contain at least one of the following: retention by time and retention by number. The rule matching module 302 is used to match the attribute information of the current timed snapshot with the directory-level retention rules and the system-level retention rules in descending order of priority during the traversal of each timed snapshot; wherein, the priority of the directory-level retention rules is higher than that of the system-level retention rules. The deletion condition identification module 303 is used to send a deletion request to the target metadata service node to which the target storage directory of the current timed snapshot belongs if the attribute information of the current timed snapshot matches the retention conditions in the current matching retention rules, so that the target metadata service node can perform the corresponding snapshot deletion operation; if the attribute information of the current timed snapshot does not match any retention conditions in the current matching retention rules, it continues to match with the retention rules of the next priority.

[0062] The distributed file system timed snapshot deletion device mentioned above is described from the perspective of functional modules. Furthermore, this invention also provides an electronic device, described from a hardware perspective. This electronic device includes a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-described distributed file system timed snapshot deletion method embodiments.

[0063] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the timed snapshot deletion method for a distributed file system.

[0064] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0065] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the timed snapshot deletion method for a distributed file system.

[0066] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above embodiments of the timed snapshot deletion method for a distributed file system.

[0067] Finally, this invention also provides a distributed file system, see [link to related document]. Figure 4 It may include a metadata service management node 401 and multiple metadata service nodes 402 connected to it.

[0068] The metadata service management node 401 is configured to execute the steps in any of the above embodiments of the distributed file system's timed snapshot deletion method. Each time a timed snapshot deletion request is sent to the metadata service node 402, the index is traversed to point to the next timed snapshot, and the corresponding matching operation is performed. While the metadata service node performs the snapshot deletion operation, it processes the matching and deletion requests for other timed snapshots in parallel, without waiting for the metadata service node to return a response. Upon receiving a timed snapshot deletion request from the metadata service management node 401, each metadata service node 402 executes the corresponding timed snapshot deletion operation, responds to the metadata service management node 401 with the snapshot deletion result, and simultaneously synchronizes the snapshot deletion result to other metadata service nodes 402 in the distributed file system.

[0069] For example, such as Figure 5As shown, the retention rules for scheduled snapshots are stored in the MDS0 metadata service. MDS0 is responsible for checking these rules. Automatic deletion of scheduled snapshots is also checked periodically by a timer, for example, every 5 seconds. MDS0 prioritizes traversing all snapshots, excluding manually created snapshots, only checking the scheduled snapshot type. When a scheduled snapshot is created, the timing policy type (interval, daily, weekly, monthly, etc.) is recorded. If the retention rules for the directory of a scheduled snapshot have been deleted (i.e., the snapshot exists, but all retention rules for the directory have been deleted), the file system's configured retention rules for scheduled snapshots are checked directly. If the retention rules for the directory of a scheduled snapshot exist, but the timing policy for creating the snapshot is unknown (this situation exists during upgrades, i.e., if older system snapshots do not record the timing creation policy type information), the file system's configured retention rules for scheduled snapshots will be checked. If the retention rules for the directory of a scheduled snapshot also exist, but no retention rule exists for the policy type recorded for that snapshot (e.g., the snapshot was created with an interval policy, but the policy was changed to a daily policy after creation), the file system's configured retention rules for scheduled snapshots are checked directly. If the directory containing the scheduled snapshot has a retention rule for that specific policy type, the process first checks if a retention period is configured. If so, it checks if the snapshot has expired. This involves comparing the snapshot's creation time with the retention period for that policy type and the current time. If the difference is less than or equal to the current time, the snapshot has expired, and the snapshot deletion process is executed. Otherwise, the process continues to check the retention count rule. If a retention count rule is configured, the process first counts all scheduled snapshots for that directory, specifically the number of snapshots created under the interval policy, the daily policy, the weekly policy, and the monthly policy. For example, if the snapshot was created under the interval policy, and the number of snapshots created under the interval policy exceeds the retention count configured for that policy type (the process is the same for other types), then the snapshot deletion process is executed. Otherwise, if the snapshot has neither expired nor exceeded the retention count, the system directly checks the scheduled snapshot retention rules configured in the file system. The file system provides a unified configuration for the retention (expiration) time and the number of times to retain timed snapshots. If the deletion conditions are not met after checking the different retention rules of the timed policy, the process will be executed again for the following check: Snapshot expiration time check: The timed snapshot itself records its creation time. The creation time of the timed snapshot is added to the retention time and compared with the current time. If it is greater than the current time, it has not expired. If it is less than or equal to the current time, it means that the snapshot has expired. Then, the snapshot directory belongs to the MDS, and MDS0 sends the deletion of the timed snapshot to the MDS that belongs to it.Snapshot retention count: If the expiration time is not configured or does not expire, check if the maximum number of snapshots for this directory exceeds the limit. If it does, delete them automatically. Otherwise, check again if the maximum number of snapshots for this file system exceeds the limit. If it does, delete them automatically. Otherwise, retain them and continue to the next round of checks.

[0070] If the automatic deletion conditions are met, MDS0 sends the snapshot to the MDS to which the directory belongs for scheduled snapshot deletion. To avoid excessive load on storage during snapshot deletion, flow control is supported, allowing a maximum of N snapshots to be deleted at a time. Furthermore, to ensure the accuracy of the retention rule, scheduled snapshot deletion is not allowed if any scheduled snapshot creation or deletion is incomplete. After other MDSs complete the deletion of the scheduled snapshot, MDS0 responds, and MDS0 continues with the next snapshot check. Snapshots are consistent across all MDSs in the distributed file system in real time; that is, snapshot deletion by other MDSs will be synchronized to MDS0, ensuring that each MDS receives the same full set of snapshot information at any given time. This mechanism is implemented through distributed transactions.

[0071] As can be seen from the above, the distributed file system in this embodiment not only supports the automatic creation of policy-based timed snapshots, but also supports configuring different retention rules for different types of timed snapshots in different directories according to the retention policy configured by the user. At the same time, in order to simplify the operation, it also supports setting file system-level retention rules and flow control during the deletion process. In order to determine the accuracy of the number of snapshots, only one snapshot is allowed to be deleted at a time in a directory. The deletion request is constructed by MDS0, and the directory-owning MDS executes the deletion of expired, useless or exceeding the retention policy snapshots, thereby realizing the automated and flexible management of the entire life cycle of snapshots.

[0072] The foregoing has provided a detailed description of a distributed file system, its timed snapshot deletion method, and the electronic device provided by this invention. The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Whether the units and algorithm steps of the various examples described in the disclosed embodiments are executed by electronic hardware or computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, and such implementations should not be considered beyond the scope of this invention. Several improvements and modifications can be made to this invention without departing from the principles of this invention, and these improvements and modifications also fall within the protection scope of this invention.

Claims

1. A method for timed snapshot deletion in a distributed file system, characterized in that, include: Configure directory-level retention rules corresponding to at least one timed snapshot strategy for different storage directories of the distributed file system, and configure system-level retention rules for the distributed file system. The directory-level retention rules and the system-level retention rules each include at least one of time-based retention conditions and number-based retention conditions. During the traversal of each timed snapshot, the attribute information of the current timed snapshot is matched with the directory-level retention rule and the system-level retention rule in descending order of priority; wherein, the directory-level retention rule has a higher priority than the system-level retention rule. If the attribute information of the current timed snapshot matches the retention conditions in the currently matched retention rule, a request to delete the timed snapshot is sent to the target metadata service node to which the target storage directory to which the current timed snapshot belongs, so that the target metadata service node performs the corresponding snapshot deletion operation; if the attribute information of the current timed snapshot does not match any retention conditions in the currently matched retention rule, then it continues to match with the retention rule of the next priority.

2. The method for timed snapshot deletion in a distributed file system according to claim 1, characterized in that, The attribute information of the current timed snapshot is matched sequentially with the directory-level retention rules and the system-level retention rules in descending order of priority, including: Based on the storage directory identifier and timed snapshot policy type of the current timed snapshot, the unique corresponding directory-level retention rule is located from the directory-level retention rules configured for different storage directories and corresponding to different timed snapshot policies. The current timed snapshot's timed snapshot policy type is an interval timed snapshot policy. Based on the storage directory identifier, the interval policy directory-level retention rule corresponding to the storage directory identifier is located from the directory-level retention rules configured for different storage directories and corresponding to the interval timed snapshot policy. The current timed snapshot's snapshot policy type is a periodic timed snapshot policy. Based on the storage directory identifier, the directory-level retention rule corresponding to the storage directory identifier and matching the periodic timed snapshot policy is located from the directory-level retention rules configured for different storage directories and corresponding to the periodic timed snapshot policy. The periodic type includes at least one of the following: daily policy, weekly policy, and monthly policy.

3. The method for timed snapshot deletion in a distributed file system according to claim 1, characterized in that, The attribute information of the current timed snapshot is matched sequentially with the directory-level retention rules and the system-level retention rules in descending order of priority, including: If the directory-level retention rule includes a time-based retention condition, and the sum of the creation time of the current timed snapshot and the retention time set by the time-based retention condition is less than or equal to the current time, then a successful match is determined. If the directory-level retention rule includes a retention condition by number, the number of all timed snapshots with the same timed policy type under the target storage directory is counted. If the counted number of relevant timed snapshots exceeds the upper limit set by the retention condition by number, a successful match is determined.

4. The method for timed snapshot deletion in a distributed file system according to claim 1, characterized in that, The attribute information of the current timed snapshot is matched sequentially with the directory-level retention rules and the system-level retention rules in descending order of priority, including: If the system-level retention rule includes a unified snapshot expiration time, and the sum of the creation time of the current timed snapshot and the unified snapshot expiration time is less than or equal to the current time, then a successful match is determined. If the system-level retention rule includes the maximum number of snapshots in the directory, the total number of snapshots under the target storage directory to which the current timed snapshot belongs is counted. If the total number of local snapshots obtained exceeds the maximum number of snapshots in the directory, then the match is determined to be successful. If the system-level retention rule includes a global maximum number of snapshots, the total number of snapshots under the distributed file system is counted. If the total number of global snapshots exceeds the global maximum number of snapshots, a successful match is determined.

5. The method for timed snapshot deletion of a distributed file system according to any one of claims 1 to 4, characterized in that, Before matching the attribute information of the current timed snapshot, which is traversed in descending order of priority, with the directory-level retention rules and the system-level retention rules, the process also includes: If there are incomplete snapshot creation operations in the target storage directory to which the current timed snapshot belongs, the matching operation for the current timed snapshot is paused. If there are no incomplete snapshot creation operations in the target storage directory to which the current timed snapshot belongs, and if the directory-level retention rule of the target storage directory to which the current timed snapshot belongs has been completely deleted, then skip the matching operation with the directory-level retention rule and directly match with the system-level retention rule; If the directory-level retention rules of the target storage directory to which the current timed snapshot belongs have not been deleted entirely, and there is no retention rule in the directory-level retention rules of the target storage directory to which the current timed snapshot belongs that corresponds to the timed snapshot policy type of the current timed snapshot, then the matching operation with the directory-level retention rules is skipped, and the system-level retention rules are matched directly. If there is a retention rule in the directory-level retention rules of the target storage directory to which the current timed snapshot belongs that corresponds to the timed snapshot policy type of the current timed snapshot, and if the current timed snapshot does not record a corresponding timed snapshot policy type, then the matching operation with the directory-level retention rule is skipped, and the system-level retention rule is directly matched.

6. The method for timed snapshot deletion of a distributed file system according to any one of claims 1 to 4, characterized in that, Before sending a delete snapshot request to the target metadata service node to which the target storage directory of the current snapshot belongs, the method further includes: Set a directory deletion flag for the target storage directory to which the current timed snapshot belongs. The directory deletion flag is used to prevent the issuance of a new snapshot deletion request to the target storage directory while there are incomplete snapshot deletion operations in the target storage directory. Upon receiving the snapshot deletion result response from the target metadata service node, the directory deletion mark is cleared.

7. The method for timed snapshot deletion of a distributed file system according to any one of claims 1 to 4, characterized in that, Also includes: During the process of traversing all timed snapshots, a flow control counter is set. The initial value of the flow control counter is zero. Whenever a request to delete a timed snapshot is sent to the metadata service node, the count value of the flow control counter is incremented by one. After each increment of the flow control counter, the incremented count value is compared with a preset flow control threshold. When the incremented count value reaches the flow control threshold, the current traversal position is recorded and the current traversal is terminated. When the next preset time period arrives, the traversal operation continues from the recorded traversal position.

8. The method for timed snapshot deletion of a distributed file system according to any one of claims 1 to 4, characterized in that, Also includes: During the process of traversing all timed snapshots, the current load status of each storage pool in the distributed file system is obtained. The current load status includes at least one or more of the following: the input / output queue depth of the storage pool, the average CPU utilization of the data nodes of the storage pool, and the average memory utilization of the data nodes of the storage pool. Based on the current load status of each storage pool, the flow control threshold corresponding to each storage pool is dynamically adjusted. Specifically, for the first storage pool whose current load status exceeds the preset high load threshold, the flow control threshold value corresponding to the first storage pool is decreased, and for the second storage pool whose current load status is lower than the preset low load threshold, the flow control threshold value corresponding to the second storage pool is increased. Before sending a request to delete a timed snapshot to the target metadata service node, determine the target storage pool corresponding to the target storage directory to which the current timed snapshot belongs, and obtain the current flow control threshold of the target storage pool; When the cumulative number of timed snapshot deletion requests sent to the target storage pool during this traversal reaches the current flow control threshold of the target storage pool, the sending of timed snapshot deletion requests to the target storage pool will stop, and the processing of timed snapshots corresponding to other storage pools will continue.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the timed snapshot deletion method of the distributed file system as described in any one of claims 1 to 8 when executing the computer program.

10. A distributed file system, characterized in that, This includes a metadata service management node and multiple metadata service nodes connected to it; The metadata service management node is configured to execute the steps of the timed snapshot deletion method of the distributed file system according to any one of claims 1 to 8. After sending a timed snapshot deletion request to the metadata service node, it will traverse the index to point to the next timed snapshot and perform the corresponding matching operation. While the metadata service node performs the snapshot deletion operation, it will process the matching and deletion requests of other timed snapshots in parallel without waiting for the metadata service node to return the response result. Upon receiving a request to delete a scheduled snapshot from the metadata service management node, each metadata service node executes the corresponding scheduled snapshot deletion operation, responds to the metadata service management node with the snapshot deletion result, and simultaneously synchronizes the snapshot deletion result to other metadata service nodes in the distributed file system.

Images