Data component deletion method and system, computer-readable storage medium, and device

Abstract

This disclosure relates to the field of distributed data storage technology, and provides a data component deletion method, a data component deletion system, a computer-readable storage medium, and an electronic device. The method includes: when a first data component is first marked for deletion, establishing a snapshot of participating nodes to coordinate the deletion operation of the first data component; during garbage collection, any participating node reads the participating node snapshot and the deletion mark bits corresponding to each participating node within the snapshot; for a target participating node whose deletion mark bits are not set in the participating node snapshot, if the target participating node is determined to be invalid, setting the deletion mark bits corresponding to the target participating node; when the deletion mark bits corresponding to all participating nodes in the participating node snapshot are set, deleting the first data component. This disclosure ensures the safe and efficient deletion of data components even in abnormal scenarios such as node failures, avoiding the remnants of orphan files.

Description

Technical Field

[0001] This disclosure relates to the field of distributed data storage technology, and in particular to a data component deletion method, a data component deletion system, a computer-readable storage medium, and an electronic device. Background Technology

[0002] With the widespread adoption of cloud-native databases, read / write separation and shared storage architectures (where multiple compute nodes share the same distributed file system, such as CubeFS, Ceph FS, and Amazon EFS) have become mainstream choices for reducing storage costs and improving elastic scalability. In this architecture, data parts (i.e., a set of ordered, immutable files in a database table) no longer belong to the private storage of a single node, but are shared and accessed by all nodes through the same file system path. Therefore, how to safely and promptly physically delete a part when it is no longer served by any node becomes a key issue in engineering practice.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure. Summary of the Invention

[0004] The purpose of this disclosure is to provide a data component deletion method, a data component deletion system, a computer-readable storage medium, and an electronic device, thereby overcoming, to at least a certain extent, the technical problem of being unable to safely and promptly delete useless data components due to limitations in related technologies.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to a first aspect of this disclosure, a method for deleting data components is provided, comprising: When the first data component is first marked for deletion, a snapshot of the participating nodes is created to coordinate the deletion operation of the first data component. During the garbage collection process, any participating node reads the snapshot of the participating node and the deletion flag bit corresponding to each participating node in the snapshot; For a target participating node whose deletion flag is not set in the participating node snapshot, if the target participating node is determined to be invalid, the deletion flag corresponding to the target participating node is set. The first data component is deleted when all the deletion flag bits corresponding to the participating nodes in the participating node snapshot have been set.

[0007] In an exemplary embodiment of this disclosure, the determination of whether the target participating node has failed is based on the following method: Obtain the timestamp of the most recent heartbeat information reported by the target participating node; Calculate the time difference between the timestamp and the current timestamp; If the time difference is greater than the failure determination time, the target participating node is determined to be failed.

[0008] In an exemplary embodiment of this disclosure, the failure determination duration is determined based on the following method: Get the mean and standard deviation of the duration of the K most recent historical garbage collections; K is an integer greater than 1; The intermediate duration is calculated based on the mean, the standard deviation, and the preset weighting coefficient. The intermediate duration is compared with the specified lower limit of duration, and the larger of the two values ​​is taken as the failure determination duration.

[0009] In an exemplary embodiment of this disclosure, setting the deletion marker bit corresponding to the target participating node includes: Read the first metadata node and its current version number of the first data component from the distributed coordination service; Based on the current version number, the updated metadata is submitted to the distributed coordination service through an atomic write operation, wherein the deletion flag bit corresponding to the target participating node has been set in the updated metadata; If the submission fails, the atomic write operation will be retried based on the latest version number.

[0010] In an exemplary embodiment of this disclosure, the method further includes: When the target participating node restarts and enters the data loading phase, for the target participating node as the second data component associated with the participating node, query the corresponding second metadata node in the distributed coordination service; If the second metadata node is not found, the loading of the second data component is skipped.

[0011] In an exemplary embodiment of this disclosure, the method further includes: During subsequent garbage collection, if a physical directory corresponding to the second data component is detected in the local shared storage, the physical directory is deleted.

[0012] In an exemplary embodiment of this disclosure, the method further includes: If the second metadata node corresponding to the second data component is found to exist, and the deletion flag bit of some participating nodes in the snapshot of the participating nodes corresponding to the second data component is set, then a proxy data component object is created for the second data component. The proxy data component object is added to the list of obsolete data components, and the second data component is deleted when all deletion flags are subsequently detected to be set.

[0013] In an exemplary embodiment of this disclosure, the method further includes: If the second metadata node corresponding to the second data component is found to exist, and the deletion flag bit of all participating nodes in the snapshot of the participating nodes corresponding to the second data component is not set, then the second data component is loaded into an active state so that the second data component can provide query services to the outside world.

[0014] In an exemplary embodiment of this disclosure, the method further includes: When the target participating node resumes operation without undergoing a restart, the following operations are performed on the proxy data component object that exists locally on the target participating node: Query the distributed coordination service to see if the first metadata node of the first data component corresponding to the proxy data component object exists; If the first metadata node is not found, check if there is a physical directory in the shared storage corresponding to the first data component; If it exists, delete the physical directory and release the proxy data component object.

[0015] According to a second aspect of this disclosure, a data component deletion system is provided, comprising: The snapshot generation module is used to create a snapshot of the first participating node for coordinating the deletion operation of the first data component when the first data component is first marked for deletion. The reading module is used to allow any participating node to read the first participating node snapshot and the deletion flag bits corresponding to each participating node in the first participating node snapshot during the garbage collection process. The setting module is used to delete target participating nodes whose flag bits are not set in the first participating node snapshot. If the target participating node is determined to be invalid, the deletion flag bit corresponding to the target participating node is set. The deletion module is used to delete the first data component when the deletion flag bits corresponding to all the participating nodes in the first participating node snapshot have been set.

[0016] According to a third aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the data component deletion method described in the first aspect.

[0017] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the data component deletion method described in the first aspect by executing the executable instructions.

[0018] As can be seen from the above technical solutions, the data component deletion method, data component deletion system, computer-readable storage medium, and electronic device in the exemplary embodiments of this disclosure have at least the following advantages and positive effects: In some embodiments of the present disclosure, when a deletion flag is first added to the first data component, a snapshot of the participating nodes is established to coordinate the deletion operation of the first data component. During garbage collection, any participating node reads the participating node snapshot and the deletion flag bits corresponding to each participating node in the participating node snapshot. For a target participating node whose deletion flag bits are not set in the participating node snapshot, if the target participating node is determined to be invalid, the deletion flag bits corresponding to the target participating node are set. When the deletion flag bits corresponding to all participating nodes in the participating node snapshot are set, the first data component is deleted. On the one hand, by fixing the snapshot of participating nodes at the beginning of the deletion process and using it as the sole basis for all subsequent setting and deletion decisions, ambiguity in the deletion status caused by dynamic changes in cluster members is avoided, ensuring the consistency and reliability of distributed deletion operations. On the other hand, when some participating nodes are unable to actively complete the deletion confirmation due to faults, network partitions, or other reasons, healthy nodes can determine their failure based on the heartbeat mechanism and set them on their behalf, thereby removing the dependency blockage of the deletion process. This allows the system to continue to advance the cleanup task even in abnormal scenarios, significantly improving the system's fault tolerance and self-healing capabilities. Furthermore, the final deletion of metadata and physical files is only triggered after all nodes in the snapshot (including failed nodes that have been set on their behalf) have completed logical confirmation. This avoids the accidental deletion of active data that is still in service and ensures that obsolete data is completely reclaimed, effectively preventing storage resource leakage and orphan file residue, and improving the security and resource utilization efficiency of the storage system.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1A flowchart illustrating the data component deletion method in an embodiment of this disclosure is shown; Figure 2 This diagram illustrates the process of determining the failure of a target participating node in an embodiment of this disclosure. Figure 3 This is a flowchart illustrating how the deletion marker position is set for the target participating node in an embodiment of this disclosure; Figure 4 This illustration shows a flowchart of the process by which a target participating node aligns its local data components with the distributed metadata state when it enters the data loading phase after a restart, according to an embodiment of this disclosure. Figure 5 This illustration shows a process diagram of cleaning up residual proxy data component objects in the memory of participating nodes that have resumed operation without undergoing a restart, according to an embodiment of this disclosure. Figure 6 This diagram illustrates the process by which a target participating node performs data loading and state alignment after a restart, as shown in an embodiment of this disclosure. Figure 7 A schematic diagram of the four-state observation state machine of the data component in an embodiment of this disclosure is shown; Figure 8 This diagram illustrates the overall architecture of the data component deletion method in an embodiment of this disclosure. Figure 9 This diagram illustrates the overall flow of the data component deletion method in an embodiment of this disclosure. Figure 10 This diagram illustrates the structure of a data component deletion system in an exemplary embodiment of this disclosure. Figure 11 A schematic diagram of the structure of an electronic device in an exemplary embodiment of this disclosure is shown. Detailed Implementation

[0022] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0023] The terms “a,” “an,” “the,” and “the” are used in this specification to indicate the presence of one or more elements / components / etc.; the terms “including” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first” and “second” are used only as markings and are not a limitation on the number of objects.

[0024] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0025] Currently, common data deletion schemes in distributed storage systems mainly fall into the following three categories: The first type is Per-Replica Local Deletion, exemplified by ClickHouse's standard ReplicatedMergeTree engine. In this mode, each compute node has independent local storage, and the physical deletion of Part files is decided autonomously by the node itself, without cross-node coordination. Nodes synchronize metadata through distributed log services such as ZooKeeper, but file-level cleanup is entirely local: when a Part is discarded due to being overwritten by a newly generated Part, the node directly deletes its corresponding directory from its local disk after a preset protection window, without waiting for confirmation from other nodes. This solution is suitable for multi-replica homogeneous deployments with non-shared storage (i.e., file systems isolated from each node), but it cannot be applied to scenarios where multiple nodes share the same file system path—if multiple nodes concurrently access the same physical path, the autonomous deletion by any node will cause other nodes to fail to read, leading to a "data stampede" problem.

[0026] The second category is distributed object deletion based on reference counting. Typical examples include HDFS DataNode's block deletion mechanism (based on Block Report) and CephRADOS's object lifecycle management. The system maintains a global reference count for each data object: the count is incremented when a node holds a reference and decremented when it is released; physical deletion is only triggered when the count reaches zero. However, this mechanism has significant limitations: on the one hand, updating the reference count requires strong consistency guarantees, which can easily become a performance bottleneck in high-concurrency scenarios; on the other hand, if a node crashes abnormally before dereferencing, the count will not decrease, leading to a "reference count leak," causing objects to remain permanently and become unrecoverable.

[0027] The third category is serialized deletion via distributed lock. This is exemplified by ClickHouse's zero-copy replication mechanism (enabled with allow_remote_fs_zero_copy_replication). When multiple replicas share the same Part file on remote storage (such as Amazon S3), the system uses ephemeral nodes on ZooKeeper to implement distributed mutex locks, serializing deletion operations on the same Part: the last node holding the reference performs the physical deletion when releasing the lock, while the remaining nodes only clean up local metadata references. While this approach avoids concurrent deletion conflicts, it still has several drawbacks: ① Lock nodes become global hotspots of contention, affecting scalability; ② Lock-holding node crashes may cause deadlocks or require timeout recovery, significantly increasing deletion latency; ③ Lock contention intensifies during large-scale concurrent deletions, limiting system throughput; ④ It cannot handle long-term faulty nodes—if a lock-holding node becomes unreachable and the lock is not released, the deletion process will be permanently blocked.

[0028] Therefore, the above solutions have the following common defects: when any node in the cluster fails (crash, network partition, long-term interruption, manual maintenance, etc.), the node cannot report its "deletion completed" status to the coordination layer in a timely manner, causing the deletion process of the entire Part to be blocked indefinitely.

[0029] This will lead to the following harms: Disk space continues to expand: Due to the large number of old parts generated by the merger, physical deletion cannot be performed, which occupies the shared file system storage space for a long time, eventually triggering a disk full alarm and blocking the writing of new data; Metadata scale out of control: The number of expired parts accumulates without limit, causing a surge in the load of background garbage collection (GC) threads. The number of related metadata nodes in ZooKeeper grows explosively, significantly aggravating memory pressure and continuously degrading the performance of coordination services. Recovery period "deletion storm": After the failed node comes back online, it is necessary to compensate for the massive backlog of deletion tasks, which will cause a sudden high load impact on ZooKeeper and the underlying file system, and trigger a secondary risk to system stability. Increased operational costs and risks: In the absence of an automated fallback mechanism, operations and maintenance personnel must manually identify zombie nodes and their remaining blocked deletion items. This operation is complex, time-consuming, and prone to human error in deletion or omission, further threatening data consistency and service availability.

[0030] This disclosure aims to solve the technical problem of long-term blocking of the data part deletion process due to node failure in multi-node database systems under shared storage architecture, and proposes a decentralized secure deletion protocol that does not require the participation of faulty nodes and allows any healthy node in the cluster to collaboratively complete the deletion decision.

[0031] Specific technical objectives include: Eliminate fault dependencies: Change the deletion trigger condition from "all N replica nodes have explicitly confirmed" to "all nodes in the Active Member Snapshot (AMS) have confirmed". This snapshot is persisted when the data component is first marked for deletion and only includes participating nodes that are in service at that time, thereby completely eliminating the blocking of the deletion process by failed nodes.

[0032] Decentralized proxy setting: This mechanism allows any healthy node to proactively check the heartbeat lease status of nodes that have not been set in the snapshot during its local garbage collection (GC) process. If it determines that a node has expired, it sets the deletion mark on its behalf using ZooKeeper's version-numbered CAS (Compare-and-Set) operation. This mechanism eliminates the need for coordinator election, naturally supports concurrency safety, and avoids single points of failure.

[0033] Restart self-healing compatibility: When a node restarts and loads its local Part, if it finds that the corresponding metadata node no longer exists in the distributed coordination service (ZNONODE), it skips loading the Part, treating it as if it has been safely deleted. This behavior reuses existing startup and recovery logic, requiring no new ZK paths or special markers, achieving zero-intrusion self-healing.

[0034] Dead node coverage guarantee: For "dead" nodes that have not been restarted but have been disconnected for a long time (such as when a process is suspended and then resumed), a file system double check mechanism is introduced in its GC thread - if the metadata node has disappeared but the physical directory still exists, the residual files are actively cleaned up to ensure eventual consistency.

[0035] Adaptive fault-tolerant window: Based on the actual time consumption statistics (mean and standard deviation) of the most recent K garbage collections, the node failure judgment threshold is dynamically calculated, and combined with preset weight coefficients and lower limit constraints, an adaptive balance is achieved between the timeliness of deletion and the risk of misjudgment.

[0036] In the embodiments of this disclosure, a method for deleting data components is first provided, which at least to some extent overcomes the shortcomings of related technologies that cannot safely and timely delete useless data components.

[0037] Figure 1The flowchart of the data component deletion method in the embodiment of this disclosure is shown. The execution subject of the data component deletion method can be a distributed storage system that performs the data component deletion.

[0038] refer to Figure 1 A data component deletion method according to an embodiment of the present disclosure includes the following steps: Step S110: When the first data component is first marked for deletion, a snapshot of the participating nodes is created to coordinate the deletion operation of the first data component. Step S120: During the garbage collection process, any participating node reads the participating node snapshot and the deletion flag bit corresponding to each participating node in the participating node snapshot; Step S130: For the target participating node whose marker bit is not set in the participating node snapshot, if the target participating node is determined to be invalid, the deletion marker bit corresponding to the target participating node is set. Step S140: When the deletion flag bit corresponding to all participating nodes in the participating node snapshot has been set, delete the first data component.

[0039] exist Figure 1 In the technical solution provided by the embodiment shown, when the first data component is first marked with a deletion flag, a snapshot of the participating nodes is established to coordinate the deletion operation of the first data component; during the garbage collection process, any participating node reads the participating node snapshot and the deletion flag bit corresponding to each participating node in the participating node snapshot; for a target participating node whose deletion flag bit is not set in the participating node snapshot, if the target participating node is determined to be invalid, the deletion flag bit corresponding to the target participating node is set; when the deletion flag bits corresponding to all participating nodes in the participating node snapshot have been set, the first data component is deleted. On the one hand, by fixing the snapshot of participating nodes at the beginning of the deletion process and using it as the sole basis for all subsequent setting and deletion decisions, ambiguity in the deletion status caused by dynamic changes in cluster members is avoided, ensuring the consistency and reliability of distributed deletion operations. On the other hand, when some participating nodes are unable to actively complete the deletion confirmation due to faults, network partitions, or other reasons, healthy nodes can determine their failure based on the heartbeat mechanism and set them on their behalf, thereby removing the dependency blockage of the deletion process. This allows the system to continue to advance the cleanup task even in abnormal scenarios, significantly improving the system's fault tolerance and self-healing capabilities. Furthermore, the final deletion of metadata and physical files is only triggered after all nodes in the snapshot (including failed nodes that have been set on their behalf) have completed logical confirmation. This avoids the accidental deletion of active data that is still in service and ensures that obsolete data is completely reclaimed, effectively preventing storage resource leakage and orphan file residue, and improving the security and resource utilization efficiency of the storage system.

[0040] The following are Figure 1The specific implementation process of each step in the process will be explained in detail: Before step S110, it should be noted that the data component (Part) in this disclosure refers to a set of ordered and immutable data files in a database table. It is a common basic storage unit in columnar storage databases (such as the Part in the MergeTree series storage engine, including but not limited to the data component used by ClickHouse's ReplicatedMergeTree engine).

[0041] Specifically, a Part can be generated by one or more INSERT write operations, or by merging multiple smaller Parts in a background MERGE operation. Each Part contains a data column file sorted by the primary key, an index file, a metadata description file, etc., and is immutable once written.

[0042] When a Part is replaced by a newly generated, larger Part that covers the same data range, that Part enters an outdated state and is no longer used for querying; it exists only as discarded data awaiting cleanup. These outdated Parts are periodically scanned by the system's garbage collection (GC) mechanism, and deletion markers are set based on a distributed coordination protocol. When deletion conditions are met, any participating node performs its physical deletion.

[0043] In the shared storage architecture disclosed herein, physical files of the same Part are shared and accessed by multiple computing nodes through a unified path. Therefore, their deletion requires cross-node coordination to avoid concurrent read / write conflicts or data loss.

[0044] The participating node snapshot in this disclosure refers to a one-time, immutable record made by the system of the set of compute nodes that are active and responsible for the data component when the first data component is first marked for deletion. Its function is equivalent to the Active Member Snapshot (AMS). Specifically, this snapshot is generated at the initial moment when the data component enters the obsolescence process, and only includes valid nodes that are registered, communicative, and hold a copy of the data component at that time; it does not include nodes that are offline, have not joined, or are in a faulty state. Once established, the snapshot content is fixed and will not be updated with subsequent cluster scaling or node status changes. This participating node snapshot can serve as a static decision-making basis for subsequent deletion operations.

[0045] In step S110, when the first data component is first marked for deletion, a snapshot of the participating nodes is created to coordinate the deletion operation of the first data component.

[0046] In this step, when the first data part first enters the deletion process—for example, because it is marked as Outdated due to being overwritten by a newly merged part and begins initializing the deletion marker bitmap—the system immediately establishes a snapshot of the participating nodes used to coordinate the deletion operation of that data part, namely the Active Member Snapshot (AMS). This snapshot records the set of all healthy, active compute nodes in the cluster that hold a copy of that part at the time of the trigger, and serves as the sole static benchmark for subsequently determining whether the deletion is complete.

[0047] The AMS (Automatic Message Type) is written to the corresponding metadata node in the distributed coordination service (such as ZooKeeper) by the first node attempting to initiate the deletion operation. The writing process uses an atomic CAS (Compare-and-Set) operation with a version number to ensure that only the first write succeeds. If multiple nodes perform this operation concurrently, subsequent nodes will abandon the write and directly reuse the existing AMS if they detect a metadata version conflict (e.g., ZooKeeper returns a ZBADVERSION error), thus avoiding duplicate creation or overwriting. The AMS data can be serialized and stored in the data body of the Part metadata node as an independent field alongside the deletion flag bitmap. It can also be embedded in the extended structure of the existing metadata header without adding a new ZooKeeper path, fully compatible with the existing storage layout.

[0048] Once the AMS (Active Memory Management Set) is established, the semantics for determining whether deletion is complete change from the traditional "all N nodes have been set" to "the set of nodes recorded in the AMS has been set." Because this snapshot is immutable after creation, even if the cluster subsequently expands, shrinks, or experiences node failures, the deletion process strictly proceeds based on the initially locked static membership set. This design fundamentally decouples deletion decisions from dynamic cluster topology, effectively eliminating deletion blocking issues caused by long-term node downtime or membership changes. It ensures that obsolete data components can be safely and promptly cleaned up within a limited time, thereby guaranteeing system storage efficiency and operational stability.

[0049] In step S120, during the garbage collection process, any participating node reads the participating node snapshot and the deletion flag bit corresponding to each participating node in the participating node snapshot.

[0050] In this step, any participating node performing a garbage collection (GC) task will actively access the metadata node corresponding to the first data component in the distributed coordination service (such as ZooKeeper) to parse out the participating node snapshot (i.e., active member snapshot, AMS) established in step S110, and simultaneously read the deletion flag status corresponding to each node in the snapshot. These flags are usually serialized and stored in the metadata node along with the AMS in the form of a bitmap or Boolean mapping, and are used to record whether each participating node has completed local cleanup or has been confirmed as deletable.

[0051] This read operation is lock-free and read-only, allowing multiple nodes to execute concurrently without introducing additional coordination overhead. By obtaining a complete snapshot and marking state, the current GC node can fully grasp the global deletion progress of this data component, thereby determining whether there are any nodes that have not yet been marked. Combined with a heartbeat lease mechanism, it further identifies whether there are any expired nodes that have not been actively marked, providing a basis for subsequent automatic marking decisions. This mechanism ensures that even if some nodes are offline, healthy nodes can still advance the deletion process based on a consistent snapshot view, avoiding accidental deletion or blockage due to missing information.

[0052] In step S130, for the target participating node whose marker bit is not set in the participating node snapshot, if the target participating node is determined to be invalid, the deletion marker bit corresponding to the target participating node is set.

[0053] In this step, during its garbage collection process, any participating node reads the participating node snapshot and the deletion flag bit corresponding to each participating node in the participating node snapshot; for the target participating node whose deletion flag bit is not set, if the target participating node is determined to be invalid, then any participating node sets the deletion flag bit corresponding to the target participating node.

[0054] refer to Figure 2 , Figure 2 This diagram illustrates the process of determining the failure of a target participating node in this embodiment of the present disclosure, including steps S201-S203: In step S201, the timestamp of the most recent heartbeat information reported by the target participating node is obtained.

[0055] In this step, the Unix timestamp stored in the persistent heartbeat node corresponding to the target participating node can be read. Each participating node can refresh its own persistent heartbeat node at a fixed period (every 30 seconds by default) through the data update operation of the distributed coordination service to indicate that it is active. This scheme uses persistent nodes instead of ephemeral nodes because: ephemeral nodes are automatically deleted by ZooKeeper when their session is disconnected, which can easily lead to the misjudgment of brief network jitter or GC pauses as permanent node failure; while persistent nodes, combined with timestamp threshold judgment, can accurately identify faulty nodes with expired leases (i.e., long-term disconnection), effectively avoiding accidental deletion triggered by momentary anomalies and improving system robustness.

[0056] In step S202, the time difference between the timestamp and the current timestamp is calculated.

[0057] In this step, the current local system time of the participating node that performs the judgment can be used as the current timestamp, and the heartbeat timestamp of the target participating node obtained in step S201 can be subtracted to obtain the time difference between the two.

[0058] In step S203, if the time difference is greater than the failure determination time, the target participating node is determined to be failed.

[0059] In this step, the time difference can be compared with the preset failure determination duration; if the time difference is greater than the failure determination duration, the target participating node is determined to be a lease expired node; where a lease expired node refers to a participating node whose lease has been determined to be invalid.

[0060] The failure determination duration can be determined as follows: obtain the mean and standard deviation of the duration of the most recent K historical garbage collections; K is an integer greater than 1; calculate the intermediate duration based on the mean, standard deviation and preset weight coefficient; compare the intermediate duration with the specified lower limit of duration and take the larger value as the failure determination duration.

[0061] Specifically, the failure determination time ( It can be calculated using the following formula 1:

[0062] in, This represents the average of the actual execution times of the K most recent garbage collection cycles. The standard deviation of the execution time of the most recent K GC cycles is used to measure the degree of fluctuation in GC duration. (For example, 3 can be used, but you can configure it according to your actual needs) and (For example, 2 can be used, and you can configure it according to your actual needs) is a preset weight coefficient used to cover the jitter range of normal GC duration and prevent unnecessary proxy deletions from being triggered due to short delays; Specify a minimum duration (e.g., 60 seconds) to prevent misjudgments caused by excessively short leases when system load is extremely low.

[0063] This adaptive algorithm dynamically adjusts the lease period by comprehensively considering the average duration and fluctuations of garbage collection (GC): when GC execution time is long or fluctuates significantly, the lease timeout is extended accordingly to prevent healthy nodes from being mistakenly judged as failed due to GC blockage; conversely, when GC is stable and fast, the lease period can be shortened to improve the system's responsiveness to real failures. This mechanism significantly improves the accuracy of lease determination and the system's fault tolerance.

[0064] refer to Figure 3 , Figure 3 This illustration shows a flowchart of how to set the deletion marker position corresponding to the target participating node in this embodiment of the present disclosure, including steps S301-S303: In step S301, the first metadata node of the first data component and its current version number are read from the distributed coordination service.

[0065] In this step, a read request can be sent to the distributed coordination service to obtain the complete data content of the metadata node corresponding to the first data component. This data content includes snapshots of participating nodes, a deletion flag bitmap composed of deletion flag bits corresponding to each participating node, and the current version number of the metadata node in the distributed coordination service.

[0066] In step S302, based on the current version number, the updated metadata is submitted to the distributed coordination service through an atomic write operation. In the updated metadata, the deletion flag bit corresponding to the target participating node has been set.

[0067] In this step, the corresponding bit in the deletion flag bitmap of the target participating node can be modified from an unset state (0) to a set state (1) locally to generate updated metadata. Subsequently, using the current version number as a condition, the updated metadata is submitted to the distributed coordination service through an atomic write operation (CAS operation) with version number. Since the deletion flag bitmap is monotonic (each bit can only be set from 0 to 1), this operation is naturally idempotent. This scheme does not require leader election, and multiple healthy participating nodes can simultaneously initiate write operations on the deletion flag bits of different target participating nodes. The CAS semantics of the distributed coordination service automatically ensures concurrent correctness. Even if multiple participating nodes concurrently write the same deletion flag bit, the result is equivalent to an atomic set operation.

[0068] In step S303, if the submission fails, the atomic write operation is retried based on the latest version number.

[0069] In this step, if the distributed coordination service returns a version conflict error code (e.g., ZBADVERSION), it indicates that the metadata node has been modified by other participating nodes during the commit. At this time, step S301 is re-executed to read the first metadata node and its latest version number, and step S302 is executed again based on the latest version number until the atomic write operation is successful or the preset retry limit is reached.

[0070] Furthermore, if multiple data components need to write the deletion flag for the same expired lease node within the same garbage collection cycle, these write operations can be merged into a single network round trip through the multi-operation atomic batch interface provided by the distributed coordination service, thereby reducing the number of interactions with the distributed coordination service and improving system efficiency.

[0071] Next, refer to Figure 1 In step S140, when the deletion flag bit corresponding to all participating nodes in the participating node snapshot has been set, the first data component is deleted.

[0072] In this step, after setting the deletion flag for the target participating node, any of the participating nodes can immediately determine whether the deletion flags for all participating nodes in the participating node snapshot have been set. If this condition is met, then any of the participating nodes can perform the following operations: ① Delete the metadata node of the first data component in the distributed coordination service; ② Delete the physical file directory corresponding to the first data component on the shared file system, including recursively deleting all subfiles and subdirectories under it.

[0073] This disclosure also provides two self-healing paths for the aforementioned target participating nodes to resume operation after being determined to be in failure: one is applicable to the case where the target participating node subsequently undergoes a restart, and the other is applicable to the case where the target participating node recovers on its own without undergoing a restart (recovery from apparent failure).

[0074] In one exemplary implementation, this disclosure provides a corresponding self-healing mechanism for situations where the target participating node resumes operation after undergoing a restart. Specifically, refer to... Figure 4 , Figure 4 This illustration shows a flowchart of the process of aligning the local data component with the distributed metadata state when the target participating node enters the data loading phase after a restart in an embodiment of this disclosure, including steps S401-S402: In step S401, when the target participating node restarts and enters the data loading phase, the second metadata node corresponding to the second data component associated with the target participating node is queried in the distributed coordination service.

[0075] In this step, the target participating node can traverse and scan all physical directories of data components in the local shared storage during the startup initialization phase, enumerate all locally persisted second data components (the aforementioned first data component is one of multiple second data components); for each second data component, it can send a query command to the distributed coordination service (ZooKeeper) to check whether there is a second metadata node that matches each second data component.

[0076] In step S402, if the second metadata node is found to be non-existent, the loading of the second data component is skipped.

[0077] In this step, if the distributed coordination service returns a non-existent response code (e.g., a ZNONODE error in ZooKeeper), it indicates that the deletion process for the second data component has been fully executed by other healthy participating nodes, including: ① the deletion flag bits of all participating nodes in the deletion flag bitmap have been set; ② the second metadata node has been deleted; ③ the physical directory on the shared file system has been cleaned up. Therefore, the current target participating node will no longer load the second data component as an active object, nor will it participate in any write operations related to the second data component.

[0078] It should be noted that after step S402, during subsequent garbage collection processes, if a physical directory corresponding to the second data component is still detected in the local shared storage, the physical directory will be deleted. Specifically, although the metadata no longer exists, orphan physical files may remain due to local I / O anomalies or incomplete deletion operations. Therefore, in subsequent rounds, the garbage collection thread will actively scan the local storage, identify directories without corresponding metadata nodes, and perform recursive deletion to ensure timely release of storage space.

[0079] In one exemplary implementation, if a second metadata node corresponding to the second data component is found to exist, and the deletion flag bit of some participating nodes in the snapshot of the participating nodes corresponding to the second data component is set, a proxy data component object is created for the second data component; the proxy data component object is added to the list of obsolete data components, and the second data component is deleted when all deletion flag bits are subsequently detected to be set.

[0080] Specifically, if the second metadata node exists, but at least one participating node in its deletion flag bitmap has its deletion flag set (i.e., the deletion process has started but not yet completed), it indicates that the data component is in a state awaiting deletion. At this time, the target participating node does not load it as an active part for normal service, but instead constructs a short-lived proxy data component object, which is only used to participate in the subsequent deletion coordination process. This proxy object is added to the local list of obsolete data components and is processed periodically by the garbage collection thread. When all deletion flags in the snapshot of the participating node corresponding to the proxy object are set, the current node executes: ① delete the second metadata node in the distributed coordination service (if it has not yet been deleted by other nodes); ② delete the physical directory on the local shared storage, thereby completing the self-healing closed loop.

[0081] In one exemplary implementation, if a second metadata node is found to exist and no deletion flag is set in the deletion flag bitmap, the second data component is loaded as an active data component and participates in the read query service normally. This implementation is applicable when the second data component is still in its effective lifecycle and needs to continue to assume data service responsibilities.

[0082] In one exemplary implementation, for the scenario where the target participating node resumes operation without undergoing a restart, this disclosure performs the following operations on the locally existing proxy data component object after the target participating node recovers: refer to Figure 5 , Figure 5 This embodiment of the present disclosure illustrates the process of cleaning up residual proxy data component objects in the memory of participating nodes that have resumed operation without undergoing a restart, including steps S501-S503: In step S501, the system queries the distributed coordination service to determine whether the first metadata node of the first data component corresponding to the proxy data component object exists.

[0083] In this step, when the target participating node periodically processes the proxy data component object in memory in its garbage collection thread, it initiates an existence query request to the distributed coordination service (such as ZooKeeper) to determine whether the first metadata node associated with the proxy data component object still exists. If the distributed coordination service returns a node non-existent response code (such as ZNONODE), it indicates that the deletion process of the first data component has been advanced to the completion stage by other healthy participating nodes, including the deletion of the metadata node.

[0084] In step S502, if the first metadata node is found to be non-existent, then check whether there is a physical directory in the shared storage corresponding to the first data component.

[0085] In this step, after confirming that the first metadata node no longer exists, the locally mounted shared file system can be further scanned to check whether there is a physical directory that matches the first data component identifier; this check is used to identify orphan files left behind due to the local deletion operation not being executed or being interrupted.

[0086] In step S503, if the physical directory exists, the proxy data component object is deleted and released.

[0087] In this step, if the physical directory is detected to still exist, a recursive deletion operation is performed to clear the directory and all its subfiles and subdirectories; regardless of whether the physical directory exists, the proxy data component object is released from memory to avoid resource leaks.

[0088] Through the above dual-check mechanism, even if the target participating node has not been restarted for a long time, the physical files remaining on its local machine due to the incomplete execution of the proxy deletion process can be finally cleaned up, thereby effectively preventing orphan files from continuously occupying the shared file system storage space and ensuring the system's storage efficiency and data consistency.

[0089] refer to Figure 6 , Figure 6 This illustration shows a flowchart of the process by which the target participating node performs data loading and state alignment after restarting in an embodiment of this disclosure, including steps S601-S617: In step S601, the node is restarted.

[0090] In this step, the target participating node restarts due to reasons such as failure, maintenance, or process termination, and enters the initialization phase. After restarting, the node needs to perform consistency verification between its local storage state and the global metadata in the distributed coordination service to avoid loading deleted data components or residual orphan files.

[0091] In step S602, the Part directory on the disk is scanned to obtain the local Part set D.

[0092] In this step, the target participating node traverses all existing data component directories on its local shared storage to construct a local Part set D, which serves as the basis for subsequent comparison with the system's expected state.

[0093] In step S603, ZooKeeper is queried to obtain the set Z of all Parts that the system expects to exist.

[0094] In this step, a request is sent to the distributed coordination service to obtain a set Z of all data components that should exist in the current system. This set consists of metadata nodes of each data component and reflects a globally consistent view of active data components.

[0095] In step S604, redundant Parts on the disk are filtered out, and proxy objects are created for Parts whose deletion progress is incomplete.

[0096] In this step, the difference between set D and set Z (i.e., Dd) is calculated. Z), obtain a set of data parts that exist only on the local disk and are not registered in ZooKeeper; for these parts, if their corresponding metadata nodes have been deleted or are in a state of pending deletion, create short-lived proxy part objects to participate in the subsequent garbage collection process.

[0097] In step S605, ZooKeeper is queried in batches to obtain the current metadata status of the local Part.

[0098] In this step, for each Part in set D, a batch query request is sent to the distributed coordination service to obtain whether the corresponding metadata node exists and the status of its deletion marker bitmap.

[0099] In step S606, each Part is classified.

[0100] In this step, based on the metadata status obtained in step S605, each Part is classified into one of the following three types: ① metadata does not exist; ② metadata exists and the deletion flag is set; ③ metadata exists and the deletion flag is not set.

[0101] In step S607, determine whether table / shared_Parts exists in ZooKeeper.

[0102] In this step, we check whether the corresponding metadata node for the Part still exists in the distributed coordination service.

[0103] In step S608, if the judgment result is "ZNONODE (does not exist)", then this part is ignored and does not participate in any ZK write operation.

[0104] In this step, since the metadata node of this Part no longer exists, it means that its deletion process has been completed, so there is no need to participate in any write operations; however, in subsequent garbage collection cycles, if the corresponding physical directory is still found to exist locally, orphan file cleanup will be performed.

[0105] In step S609, if the determination result is "existing", then it is further determined whether at least one node has set the deletion flag bit.

[0106] In this step, after confirming the existence of the metadata node, its deletion flag bitmap is parsed to determine whether the deletion flag bit of at least one participating node has been set.

[0107] In step S610, if the judgment result is "yes" (i.e., a node has been set), a proxy Part object (short lifecycle type) is created, added to the discarded Part list, and the GC process continues to complete the subsequent deletion steps.

[0108] In this step, the Part is in the deletion process but has not yet been completed, so it is not loaded as an active object. Instead, a proxy Part object is constructed and added to the local list of discarded Parts.

[0109] In step S611, if the judgment result is "no" (i.e. no node is set), then the Part is loaded as an ActivePart (normal service read query).

[0110] In this step, the Part is still within the normal service cycle, so it is loaded as an active data part and registered as a copy that can respond to client read requests.

[0111] In step S612, FakePart enters the GC thread for processing.

[0112] In this step, the proxy Part objects created in step S610 are submitted to the garbage collection thread queue for further processing.

[0113] In step S613, the GC thread reads the ZKPart node.

[0114] In this step, the garbage collection thread reads the latest metadata node of the Part from the distributed coordination service to obtain its current version number and deletion marker bitmap status.

[0115] If it exists, proceed to step S614 for the normal GC process; If it does not exist (ZNONODE), proceed to step S615 to determine if the shared file system path exists. In this step, check whether the physical directory corresponding to the Part still exists in the shared storage. If it does not exist, proceed to step S617. In step S616, if the file directory exists, it is deleted.

[0116] In this step, if a physical directory is detected, a recursive deletion operation is performed to clear all its subfiles and subdirectories.

[0117] In step S617, regardless of whether the proxy object is deleted, the proxy object in memory is cleaned up, and the process ends.

[0118] In this step, the resources of the proxy Part object in memory are released to avoid long-term occupation. After all processing is completed, the relevant memory resources are released, and the data loading and state alignment process ends.

[0119] refer to Figure 7 , Figure 7 A schematic diagram of the four-state observation state machine of the data component in an embodiment of this disclosure is shown, as follows: Figure 7 As shown: During system operation, data components can be in one of four observable states: Active, ProxyPart, Ignore, and Transient. This state machine is used to uniformly manage the lifecycle of data components from normal service to final deletion, ensuring that all nodes have a consistent perception of the state of the same data component, and guiding their behavior in read queries, GC processes, and metadata operations.

[0120] (1) Active State – When a data component completes loading and is registered as an active replica, it enters the Active state. This state indicates that the data component is currently in a normal service cycle, its corresponding physical directory exists in the shared file system, and the ZooKeeper metadata node exists and has not been set with a deletion flag. Data components in this state can respond to client read requests and participate in data services.

[0121] (2) Proxy Part State – When a node detects that the ZK metadata nodes of its local Part have been deleted or some deletion flags have been set, but the entire deletion process has not yet been completed, the Part enters the Proxy Part state. This state indicates that the data part is being processed by the garbage collection mechanism, and its memory objects are only used to coordinate the deletion process and do not participate in any read queries. Triggering conditions include: after the node restarts, it finds that the ZK metadata nodes do not exist (ZNONODE), but the physical directory still exists locally; or, during the GC process, it is detected that some nodes in the deletion flag bitmap of the Part have been set. Data parts in this state will be added to the list of discarded Parts, and the GC thread will continue to advance the deletion process.

[0122] (3) Ignore State – When the GC thread detects that the ZK metadata node corresponding to the Part has returned a ZNONODE response code (i.e., the metadata node no longer exists), and its physical directory has also been cleaned up, the Part enters the Ignore state. This state indicates that the deletion process of the data part has been completed, and all its related resources have been released. Data parts in this state no longer participate in any system operations, and their memory objects are only retained until the next round of GC reclamation, after which they are automatically cleared.

[0123] (4) Transient State – When a data component's metadata node in the distributed coordination service is deleted and its physical directory is cleared, but its memory object has not been completely released, it enters the Transient state. This state is a temporary transitional state, indicating that the data component is no longer visible to the external system and exists only in local memory, awaiting final reclamation. This state does not provide services externally and does not participate in any ZK operations; its lifecycle is controlled by the memory reclamation mechanism, and it disappears after the operation is completed atomically.

[0124] State transition logic: Active → Proxy Part: When a node restarts and finds that the ZK metadata node does not exist (ZNONODE), or detects that at least one node in the deletion flag bitmap is set, this Part transitions from the Active state to the Proxy Part state.

[0125] Proxy Part → Ignore: When the GC thread confirms that the ZK metadata node has been deleted (ZNONODE) ​​and the physical directory has been cleared, the Part enters the Ignore state.

[0126] Proxy Part → Transient: When the ZK metadata nodes and physical directories have been cleaned up, but the memory objects have not yet been released, the Transient state is entered.

[0127] Ignore / Transient → Termination: In subsequent GC cycles, the memory object is actively released or automatically reclaimed, and the state terminates.

[0128] refer to Figure 8 , Figure 8 This diagram illustrates the overall architecture of the data component deletion method in an embodiment of this disclosure, such as... Figure 8 As shown: The data component deletion method is implemented in a distributed system environment, involving the collaborative work of multiple computing nodes. Through a heartbeat mechanism, synchronization of the deletion marker bitmap, and asynchronous cleanup by the GC thread, it ensures the safe and consistent deletion of data components in multi-replica scenarios. The overall architecture includes a computing layer, a distributed file system, and a metadata layer, with each component working together to complete the entire process from deletion triggering to physical resource reclamation.

[0129] The compute layer comprises multiple participating nodes (e.g., node 0, node 1, and node 2), each running an independent GC thread responsible for the lifecycle management of its local data components. The GC threads of node 0 (healthy) and node 1 (healthy) are running normally, continuously monitoring their local component status and participating in the deletion process. The GC thread of node 2 (faulty) has expired and no longer participates in deletion coordination, but its local status may still affect consistency judgments. All nodes maintain connections through a heartbeat mechanism; if a node fails to send a heartbeat for an extended period, it is considered "expired" or "unreachable."

[0130] The metadata layer consists of Part metadata nodes in ZooKeeper and the heartbeat records of each node. The Part metadata node records a deletion flag bitmap (node ​​0=1, node 1=1, node 2=0) and a snapshot of active members {0,1,2} created and locked when the Part is first marked with a deletion flag. The node heartbeat records show that node 0's most recent heartbeat was 5 seconds ago, node 1's was 8 seconds ago, and node 2's was 900 seconds ago, explicitly marked "lease expired". When node 0 detects that node 2's lease has timed out (i.e., expired), it triggers a proxy setting operation (corresponding to label ② in the diagram), setting its deletion flag to 1 to advance deletion consensus. Once the deletion flags of all nodes in the active member snapshots are set, the deletion condition is met, and the system deletes the Part metadata node in ZooKeeper (corresponding to label ③ in the diagram).

[0131] Subsequently, any healthy node (such as node 0) initiates a cleanup operation on the corresponding physical file directory in the distributed file system (as shown in ④ in the diagram), completing the final resource reclamation. The entire process ensures that even in the event of partial node failure, the system can still identify failed nodes through a heartbeat mechanism, have healthy nodes complete the deletion confirmation, and safely clear metadata and physical data after reaching a global consensus. This effectively avoids orphan file residue and guarantees the consistency, fault tolerance, and storage efficiency of the distributed storage system.

[0132] refer to Figure 9 , Figure 9 This diagram illustrates the overall flow of the data component deletion method in this embodiment, including steps S901-S920: In step S901, the GC thread is triggered.

[0133] In this step, the garbage collection thread of the target participating node is started periodically to begin cleaning up and synchronizing the status of local expired Parts; this process is independent of the main business process, ensuring the effective release of system resources.

[0134] In step S902, the list of expired Parts in memory is obtained.

[0135] In this step, the GC thread reads a list of all Part objects marked as "expired" from the memory cache of the current node, which serves as the target set for this GC process.

[0136] In step S903, each expired Part is traversed.

[0137] In this step, for each expired Part in the list, subsequent judgment and processing procedures are executed sequentially to determine whether it meets the deletion conditions.

[0138] In step S904, is it determined whether P is a regular Part?

[0139] In this step, if the determination result is "yes", the deletion process of the regular Part will be entered (steps S905–S915); if the determination result is "no" (i.e., it is a proxy Part or an expired temporary object), the process will jump to the non-regular Part processing flow (steps S916–S920).

[0140] I. Standard Part Deletion Procedure (S905–S915)

[0141] In step S905, if the judgment result is "yes" (i.e., P is a regular Part), then read the metadata record of the Part in ZooKeeper to obtain the deletion marker bitmap and the current version number.

[0142] In this step, a request is sent to the distributed coordination service to obtain the latest metadata information of this Part in ZK, including: Delete marker bitmap (records the deletion status of each participating node); Metadata version number (used to determine if a change has occurred); Active member snapshot (used to confirm the current set of valid nodes).

[0143] In step S906, the deletion marker position corresponding to this node is set to 1, and the operation is committed through an atomic write operation with a version number.

[0144] In this step, the node sets its own deletion flag to 1 and uses a CAS (Compare-and-Swap) atomic operation to write the updated deletion flag bitmap into ZK, ensuring concurrency safety.

[0145] In step S907, the heartbeat records of each node are scanned, and a snapshot of the currently active members is calculated.

[0146] In this step, the GC thread reads the most recent heartbeat timestamp of all participating nodes, builds a snapshot of the active members of the current system based on a preset timeout threshold (such as 10 seconds), and excludes expired or faulty nodes.

[0147] In step S908, it is determined whether there are any lease expired nodes in AMS with their deletion flag bit = 0.

[0148] In this step, we check if there is a node in the active member snapshot that has expired but whose deletion flag is still 0. If so, it means that the node has not yet completed the deletion process and we need to wait for it to be restored or for another node to set the flag.

[0149] If the result is "no", then proceed to step S912.

[0150] In step S909, if the judgment result is "yes", then the deletion mark position of the expired node is set to 1 by an atomic write operation with the same version number.

[0151] In this step, when an expired node is detected to have not set the deletion mark, a healthy node performs an atomic write operation to set the deletion mark to 1, ensuring that the deletion process is not blocked due to individual node failures.

[0152] In step S910, how is the operation result determined?

[0153] In this step, check whether the atomic write operation in the previous step was successfully executed; if it fails (e.g., returning ZBADVERSION), it means that the metadata has been modified by other nodes, and you need to reread the latest status and retry.

[0154] In step S911, if the operation fails (ZBADVERSION), the latest metadata and version number are reread to determine whether it is still necessary to write the code.

[0155] In this step, when the atomic write fails, the GC thread retrieves the latest metadata and version number from ZK, re-evaluates the deletion flag status, and decides whether to try the write operation again.

[0156] In step S912, if the operation is successful, it is determined whether the deletion flag bit of all nodes in the active member snapshot has been set.

[0157] In this step, we check whether the deletion flag bit of all nodes in the current active member snapshot is 1, indicating that the deletion process has reached consensus.

[0158] In step S913, if the judgment result is "yes", then delete the ZK Part metadata node.

[0159] In this step, once all valid nodes have completed the deletion confirmation, the deletion operation of the corresponding Part metadata node in ZooKeeper is triggered, completing the logical deletion.

[0160] In step S914, the corresponding physical directory in the shared file system is deleted.

[0161] In this step, after the ZK metadata is successfully deleted, any healthy node initiates a recursive deletion operation on the physical directory corresponding to that Part in the distributed file system to release storage resources.

[0162] In step S915, the next part is processed.

[0163] In this step, after completing the deletion process of the current Part, return to step S903 to continue processing the next expired Part in the list.

[0164] II. Non-standard Part Handling Procedure (S916-S920)

[0165] In step S916, the ZK Part node status is read.

[0166] In this step, the GC thread checks whether the metadata node for the Part exists in ZK. If it exists, it jumps to step S920 and waits for the next round of GC processing; if it does not exist (returning ZNONODE), it proceeds to step S917.

[0167] In step S917, it is determined whether the shared file system path still exists.

[0168] In this step, we check whether the physical directory corresponding to the Part still exists in the shared storage.

[0169] In step S918, if the orphan file directory exists, it is deleted.

[0170] If any physical directory remnants are found in this step, a recursive deletion operation will be performed to remove orphan files and prevent storage leaks.

[0171] In step S919, only the Part objects in memory are removed.

[0172] In this step, the proxy object or cache structure of the Part in memory is released, and resource reclamation is completed.

[0173] In step S920, skip (wait for the next round of GC).

[0174] In this step, if the ZK node still exists, no operation is performed, and it is checked again in a subsequent GC round to avoid duplicate processing.

[0175] Based on the above technical solutions, this disclosure has at least the following technical effects: First, it significantly improves the availability and efficiency of the distributed deletion process.

[0176] By changing the deletion consensus condition from "confirmation by all original participating nodes" to "confirmation by all currently valid nodes in the Active Member Snapshot (AMS)," and allowing healthy nodes to set the deletion flag for failed nodes, the deletion blocking problem caused by permanent node failure or long-term offline status is completely eliminated, ensuring that data components can be cleaned up within the expected time and avoiding continuous occupation of storage resources.

[0177] Second, implement a decentralized, high-concurrency, and secure collaborative deletion mechanism.

[0178] Any healthy node can independently detect lease-expired nodes in the local GC process and complete the proxy write through ZooKeeper's versioned CAS atomic operation. This eliminates the need for master node election or a global coordinator, reducing system complexity and naturally avoiding state conflicts caused by concurrent proxy writes from multiple nodes, thus ensuring the consistency and atomicity of the deletion marker bitmap.

[0179] Third, enhance the system's self-healing ability and state consistency.

[0180] Whether it's a node restart scenario (quickly skipping the loading of deleted parts via ZNONODE) ​​or a process recovery scenario (using the GC thread to perform dual verification of ZK and the file system), the system can automatically align local state with global metadata to prevent orphan file residues or invalid data from being mistakenly loaded, thereby maintaining a strongly consistent data view in a production environment where anomalies are frequent.

[0181] Fourth, implement a dynamically adaptable fault-tolerant strategy to balance security and response speed.

[0182] By dynamically adjusting the lease timeout threshold based on historical garbage collection duration statistics, the system can adapt to the node behavior characteristics under different load and network environments. While avoiding misjudging node failure due to brief fluctuations, it can also identify real faults in a timely manner, balancing the timeliness of the deletion process with the reliability of the judgment.

[0183] In summary, this disclosure achieves a synergistic breakthrough in four dimensions: availability, consistency, decentralized collaboration, and adaptive fault tolerance, effectively solving the core pain points of "cannot be deleted, cannot be deleted accurately, and cannot be completely deleted" in shared storage multi-replica databases.

[0184] This disclosure also provides a data component deletion system. Figure 10 This diagram illustrates the structure of a data component deletion system in an exemplary embodiment of this disclosure; as shown below. Figure 10 As shown, the data component deletion system 1000 may include a snapshot generation module 1010, a reading module 1020, a setting module 1030, and a deletion module 1040. Wherein: The snapshot generation module 1010 is used to create a snapshot of the first participating node for coordinating the deletion operation of the first data component when the first data component is first marked with a deletion tag; The reading module 1020 is used to, during the garbage collection process, read the first participating node snapshot and the deletion flag bits corresponding to each of the participating nodes in the first participating node snapshot; The setting module 1030 is used to delete target participating nodes whose marker bits are not set in the first participating node snapshot. If the target participating node is determined to be invalid, the deletion marker bit corresponding to the target participating node is set. The deletion module 1040 is used to delete the first data component when the deletion flag bits corresponding to all the participating nodes in the first participating node snapshot have been set.

[0185] In an exemplary embodiment of this disclosure, the setting module 1030 determines whether the target participating node has failed based on the following method: Obtain the timestamp of the most recent heartbeat information reported by the target participating node; Calculate the time difference between the timestamp and the current timestamp; If the time difference is greater than the failure determination time, the target participating node is determined to be failed.

[0186] In an exemplary embodiment of this disclosure, the setting module 1030 determines the failure determination duration based on the following method: Get the mean and standard deviation of the duration of the K most recent historical garbage collections; K is an integer greater than 1; The intermediate duration is calculated based on the mean, the standard deviation, and the preset weighting coefficient. The intermediate duration is compared with the specified lower limit of duration, and the larger of the two values ​​is taken as the failure determination duration.

[0187] In an exemplary embodiment of this disclosure, the setting module 1030 sets the deletion marker corresponding to the target participating node, including: Read the first metadata node and its current version number of the first data component from the distributed coordination service; Based on the current version number, the updated metadata is submitted to the distributed coordination service through an atomic write operation, wherein the deletion flag bit corresponding to the target participating node has been set in the updated metadata; If the submission fails, the atomic write operation will be retried based on the latest version number.

[0188] In an exemplary embodiment of this disclosure, the deletion module 1040 is configured to: When the target participating node restarts and enters the data loading phase, for the target participating node as the second data component associated with the participating node, query the corresponding second metadata node in the distributed coordination service; If the second metadata node is not found, the loading of the second data component is skipped.

[0189] In an exemplary embodiment of this disclosure, the deletion module 1040 is configured to: During subsequent garbage collection, if a physical directory corresponding to the second data component is detected in the local shared storage, the physical directory is deleted.

[0190] In an exemplary embodiment of this disclosure, the deletion module 1040 is configured to: If the second metadata node corresponding to the second data component is found to exist, and the deletion flag bit of some participating nodes in the snapshot of the participating nodes corresponding to the second data component is set, then a proxy data component object is created for the second data component. The proxy data component object is added to the list of obsolete data components, and the second data component is deleted when all deletion flags are subsequently detected to be set.

[0191] In an exemplary embodiment of this disclosure, the deletion module 1040 is configured to: If the second metadata node corresponding to the second data component is found to exist, and the deletion flag bit of all participating nodes in the snapshot of the participating nodes corresponding to the second data component is not set, then the second data component is loaded into an active state so that the second data component can provide query services to the outside world.

[0192] In an exemplary embodiment of this disclosure, the deletion module 1040 is configured to: When the target participating node resumes operation without undergoing a restart, the following operations are performed on the proxy data component object that exists locally on the target participating node: Query the distributed coordination service to see if the first metadata node of the first data component corresponding to the proxy data component object exists; If the first metadata node is not found, check if there is a physical directory in the shared storage corresponding to the first data component; If it exists, delete the physical directory and release the proxy data component object.

[0193] The specific details of each module in the aforementioned data component deletion system have been described in detail in the corresponding data component deletion methods, so they will not be repeated here.

[0194] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0195] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0196] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0197] This disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device.

[0198] Computer-readable storage media can be, for example—but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0199] A computer-readable storage medium can be sent, propagated, or transmitted for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0200] A computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0201] Furthermore, this disclosure also provides an electronic device capable of implementing the above-described method.

[0202] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0203] The following reference Figure 11 To describe an electronic device 1100 according to such an embodiment of the present disclosure. Figure 11 The electronic device 1100 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0204] like Figure 11 As shown, the electronic device 1100 is presented in the form of a general-purpose computing device. The components of the electronic device 1100 may include, but are not limited to: at least one processor 1110, at least one memory 1120, a bus 1130 connecting different system components (including memory 1120 and processor 1110), and a display 1140.

[0205] The memory stores program code that can be executed by the processor 1110, causing the processor 1110 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processor 1110 can perform actions such as... Figure 1As shown: Step S110, when the first data component is first marked with a deletion flag, a snapshot of the participating nodes is established to coordinate the deletion operation of the first data component; Step S120, during garbage collection, any participating node reads the participating node snapshot and the deletion flag bit corresponding to each participating node in the participating node snapshot; Step S130, for a target participating node whose deletion flag bit is not set in the participating node snapshot, if the target participating node is determined to be invalid, the deletion flag bit corresponding to the target participating node is set; Step S140, when the deletion flag bits corresponding to all participating nodes in the participating node snapshot have been set, the first data component is deleted.

[0206] The memory 1120 may include a readable medium in the form of volatile storage, such as random access memory (RAM) 11201 and / or cache memory 11202, and may further include read-only memory (ROM) 11203.

[0207] The memory 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0208] Bus 1130 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0209] Electronic device 1100 can also communicate with one or more external devices 1200 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with the electronic device 1100, and / or with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1150. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1160. As shown, network adapter 1160 communicates with other modules of electronic device 1100 via bus 1130. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0210] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A data component deletion method characterized by comprising: include: When the first data component is first marked for deletion, a snapshot of the participating nodes is created to coordinate the deletion operation of the first data component. During the garbage collection process, any participating node reads the snapshot of the participating node and the deletion flag bit corresponding to each participating node in the snapshot; For a target participating node whose deletion flag is not set in the participating node snapshot, if the target participating node is determined to be invalid, the deletion flag corresponding to the target participating node is set. The first data component is deleted when all the deletion flag bits corresponding to the participating nodes in the participating node snapshot have been set.

2. The method according to claim 1, characterized in that, The following method is used to determine whether the target participating node has failed: Obtain the timestamp of the most recent heartbeat information reported by the target participating node; Calculate the time difference between the timestamp and the current timestamp; If the time difference is greater than the failure determination time, the target participating node is determined to be failed.

3. The method according to claim 2, characterized in that, The failure determination time is determined based on the following method: Get the mean and standard deviation of the duration of the K most recent historical garbage collections; K is an integer greater than 1; The intermediate duration is calculated based on the mean, the standard deviation, and the preset weighting coefficient. The intermediate duration is compared with the specified lower limit of duration, and the larger of the two values ​​is taken as the failure determination duration.

4. The method according to claim 1, characterized in that, The step of setting the deletion marker position corresponding to the target participating node includes: Read the first metadata node and its current version number of the first data component from the distributed coordination service; Based on the current version number, the updated metadata is submitted to the distributed coordination service through an atomic write operation, wherein the deletion flag bit corresponding to the target participating node has been set in the updated metadata; If the submission fails, the atomic write operation will be retried based on the latest version number.

5. The method according to claim 1, characterized in that, The method further includes: When the target participating node restarts and enters the data loading phase, for the target participating node as the second data component associated with the participating node, query the corresponding second metadata node in the distributed coordination service; If the second metadata node is not found, the loading of the second data component is skipped.

6. The method according to claim 5, characterized in that, The method further includes: During subsequent garbage collection, if a physical directory corresponding to the second data component is detected in the local shared storage, the physical directory is deleted.

7. The method according to claim 5, characterized in that, The method further includes: If the second metadata node corresponding to the second data component is found to exist, and the deletion flag bit of some participating nodes in the snapshot of the participating nodes corresponding to the second data component is set, then a proxy data component object is created for the second data component. The proxy data component object is added to the list of obsolete data components, and the second data component is deleted when all deletion flags are subsequently detected to be set.

8. The method according to claim 5, characterized in that, The method further includes: If the second metadata node corresponding to the second data component is found to exist, and the deletion flag bit of all participating nodes in the snapshot of the participating nodes corresponding to the second data component is not set, then the second data component is loaded into an active state so that the second data component can provide query services to the outside world.

9. The method according to claim 1, characterized in that, The method further includes: When the target participating node resumes operation without undergoing a restart, the following operations are performed on the proxy data component object that exists locally on the target participating node: Query the distributed coordination service to see if the first metadata node of the first data component corresponding to the proxy data component object exists; If the first metadata node is not found, check if there is a physical directory in the shared storage corresponding to the first data component; If it exists, delete the physical directory and release the proxy data component object.

10. A data component deletion system, characterized in that, include: The snapshot generation module is used to create a snapshot of the first participating node for coordinating the deletion operation of the first data component when the first data component is first marked for deletion. The reading module is used to allow any participating node to read the first participating node snapshot and the deletion flag bits corresponding to each participating node in the first participating node snapshot during the garbage collection process. The setting module is used to delete target participating nodes whose flag bits are not set in the first participating node snapshot. If the target participating node is determined to be invalid, the deletion flag bit corresponding to the target participating node is set. The deletion module is used to delete the first data component when the deletion flag bits corresponding to all the participating nodes in the first participating node snapshot have been set.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data component deletion method according to any one of claims 1 to 9.

12. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the data component deletion method according to any one of claims 1 to 9 by executing the executable instructions.

Images