Cluster management device and method for private interplanetary file system(IPFS)

Abstract

A cluster management device for a distributed file storage system (IPFS) according to an embodiment of the present invention comprises: a communication unit that communicates with a plurality of nodes within a cluster; a memory that stores management information including at least one instruction and a node-specific lifecycle, data distribution, and replication policy; and a processor that performs an operation according to the instruction. When a new node is added to a cluster, the processor calculates a list of data that the new node must replicate based on the data distribution and replication policy of the current cluster and performs data reorganization. When a specific node is deleted, the processor performs a data evacuation procedure to replicate data fragments stored in the node to another node and then controls the specific node to be separated from the cluster.

Description

Technology Field

[0001] The present invention relates to a private distributed file storage management technology, and more specifically, to a cluster management device and method that enable stable and efficient cluster operation by maintaining data replication consistency without service interruption even when nodes are added, deleted, or fail in a private IPFS (InterPlanetary File System)-based cluster environment, and by performing policy-based dynamic replication control and data reorganization. Background Technology

[0002] The InterPlanetary File System (IPFS) is a distributed file storage system based on content addressing that provides a decentralized storage environment by sharing and replicating data blocks among nodes without a central server. However, existing IPFS cluster structures rely on voluntary pinning by each node for replication and data distribution management, making it difficult to centrally control the consistency of replica counts and locations. Furthermore, the overall replication balance is prone to being disrupted when nodes are added or removed. Additionally, there is a high probability of service interruptions and replication inconsistencies occurring during the redistribution of the entire data in the event of cluster expansion or node failure, and there are limitations in responding to environmental changes due to static replication policies. Therefore, there is a need for technology that centrally monitors the status of all nodes within the cluster, dynamically adjusts replication factors based on policies, and performs automatic replication and data rebalancing in response to node additions, deletions, or failure events. Prior art literature

[0003] Published Patent Application No. 10-2023-0004006 The problem to be solved

[0004] The objective of the present invention is to provide a cluster management device and method for a distributed file storage system (IPFS) capable of maintaining replication consistency by centrally managing the node status and data replication status within the cluster, dynamically setting replication factors based on policies, performing efficient replication control according to data importance and usage frequency, automatically performing incremental replication and data evacuation without service interruption in the event of node addition, deletion, or failure events, guaranteeing the integrity of the replication and deletion processes, and ensuring the reliability of data governance by recording all management history in an auditable log format. means of solving the problem

[0005] A cluster management device for a distributed file storage system (IPFS) according to an embodiment of the present invention for solving the above problem comprises: a communication unit that communicates with a plurality of nodes within a cluster; a memory that stores management information including at least one instruction and a node-specific lifecycle, data distribution, and replication policy; and a processor that performs an operation according to the instruction. The processor, when a new node is added to the cluster, calculates a list of data to be replicated by the new node based on the data distribution and replication policy of the current cluster and performs data reorganization; when a specific node is deleted, performs a data evacuation procedure to replicate data fragments stored in the node to another node and then controls the specific node to be separated from the cluster. The processor applies a dynamic replication policy according to the importance or usage frequency of the data, sets a replication factor, and monitors the cluster status to automatically adjust the actual number of replicas so that it matches the policy.

[0006] In an embodiment, the processor is characterized by centrally managing the lifecycle of all nodes through a Pin orchestrator.

[0007] In an embodiment, the processor is characterized by performing automatic data distribution by issuing a data rebalancing command considering the load status of the cluster when the new node is added.

[0008] In an embodiment, the processor is characterized by identifying data fragments stored only in the node when deleting the specific node, and delaying deletion until replication to another normal node is completed.

[0009] In an embodiment, the processor is characterized by executing node separation only after confirming the completion of replication to prevent data loss during a data evacuation procedure.

[0010] delete

[0011] In an embodiment, the processor is characterized by supporting different replication factors for specific data on a file or folder basis.

[0012] In an embodiment, the processor is characterized by performing additional replication when the number of replicas is less than the policy when a node failure occurs, and deleting unnecessary replicas when the number is more than the policy.

[0013] In an embodiment, the processor is characterized by controlling operation without service interruption during data reorganization or evacuation procedures.

[0014] A cluster management method for a distributed file system according to an embodiment of the present invention for solving the above problem comprises: a step of, when a new node is added to a cluster in a processor, calculating a list of data to be replicated by the new node based on the data distribution and replication policy of the current cluster and performing data reorganization; a step of, when a specific node is deleted in the processor, performing a data evacuation procedure to replicate data fragments stored in the node to another node and then separating the specific node from the cluster; and a step of, in the processor, applying a dynamic replication policy according to the importance or usage frequency of the data, setting a replication factor, and monitoring the cluster status to automatically adjust the actual number of replicas so that it matches the policy. Effects of the invention

[0015] Therefore, the present invention can always maintain the replication factor based on policy standards by performing automatic replication and data evacuation procedures even when nodes are added, deleted, or failures occur.

[0016] The present invention can simultaneously ensure storage efficiency and stability by adjusting the replication factor in real time according to data importance and usage frequency.

[0017] The present invention can maintain data balance without service interruption and without total redistribution using an incremental rebalancing method.

[0018] The present invention can prevent data loss and falsification / tampering through CID-based integrity verification and log management functions, and can respond to external audits.

[0019] Therefore, thanks to the aforementioned advantages, administrators can minimize human intervention as replication and deletion are performed automatically simply by modifying the policy. Brief explanation of the drawing

[0020] FIG. 1 is a configuration diagram of a cluster management device for a distributed file storage system (IPFS) according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a cluster management method for a distributed file system according to an embodiment of the present invention. Figure 3 is a detailed flowchart of the S710 process shown in Figure 2. Figure 4 is a detailed flowchart of the S720 process shown in Figure 2. Figure 5 is a detailed flowchart of the S730 process shown in Figure 2. Specific details for implementing the invention

[0021] Various embodiments of the present disclosure are described below in conjunction with the accompanying drawings. As various embodiments of the present disclosure may be subject to various modifications and may have various forms, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the various embodiments of the present disclosure to specific forms, and it should be understood that they include all modifications and / or equivalents and substitutions that fall within the spirit and scope of the various embodiments of the present disclosure. Similar reference numerals have been used for similar components in connection with the description of the drawings.

[0022] In the various embodiments of the present disclosure, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0023] In various embodiments of the present disclosure, expressions such as “or” include any and all combinations of the words listed together. For example, “A or B” may include A, may include B, or may include both A and B. Expressions such as “first,” “second,” “first,” or “second” used in various embodiments of the present disclosure may modify various components of the various embodiments but do not limit such components. For example, such expressions do not limit the order and / or importance of such components and may be used to distinguish one component from another. When a component is referred to as being “connected” or “joined” to another component, it should be understood that the component may be directly connected to or joined to the other component, or that a new component may exist between the component and the other component. In embodiments of the present disclosure, terms such as “module,” “unit,” “part,” etc. are used to refer to a component that performs at least one function or operation, and such a component may be implemented in hardware or software, or a combination of hardware and software. In addition, multiple "modules," "units," "parts," etc., can be integrated into at least one module or chip and implemented as at least one processor, except where each needs to be implemented as individual specific hardware.

[0024] Terms such as those defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings in the context of the relevant technology and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the various embodiments of the present disclosure.

[0025] Hereinafter, a cluster management device and method for a distributed file storage system (IPFS) according to an embodiment of the present invention will be described in more detail based on the attached drawings.

[0026] FIG. 1 is a configuration diagram of a cluster management device for a distributed file storage system (IPFS) according to an embodiment of the present invention.

[0027] As illustrated in FIG. 1, a cluster management device (100) for a distributed file storage system (IPFS) according to an embodiment of the present invention includes a communication unit (110), a memory (120), and a processor (130).

[0028] The communication unit (110) is configured as a network interface for transmitting and receiving data and control signals bidirectionally with multiple nodes within a cluster, and operates to periodically or asynchronously exchange messages such as status information, replication commands, data fragment transmission, policy change notifications, and event logs between nodes.

[0029] The communication unit (110) includes Node Identification Management and Session Management functions suitable for the Peer-to-Peer structure of the IPFS network, and each node is mutually authenticated through a unique node ID and a public key-based addressing scheme. Accordingly, the communication unit (110) ensures the confidentiality and integrity of data transmission using Transport Layer Security (TLS) or the encryption layer of IPFS itself.

[0030] The communication unit (110) supports the cluster internal network control protocol and, in conjunction with internal modules of the processor (130), such as the Pin orchestrator (131), replication consistency management unit (133), and policy management unit (132) described later, transmits replication, deletion, and rebalancing commands to each node. In addition, it transmits status reports periodically collected from each node—namely, information such as the node's available capacity, list of stored data, connection status, CPU and memory usage—to a metadata base, thereby enabling monitoring of the status of the entire system. The communication unit (110) adopts an event-based asynchronous messaging structure when transmitting data to support parallel data exchange between nodes, and when an event occurs, such as node addition, deletion, or fault detection, it notifies the Pin orchestrator (131) of the event in real time. Furthermore, if packet loss or delay occurs during transmission, the communication unit (110) performs automatic retransmission and path switching functions to maintain the stability of the replication and data rebalancing process.

[0031] The communication unit (110) is designed to logically separate the data channel and the control channel so that the replication data transmission (Data Plane) and the management command (Control Plane) do not interfere with each other. The management command is transmitted preferentially through the control channel, and if a response delay or failure state of each node is detected, the communication unit (110) automatically updates the retry queue or bypass node list to prevent loss of the replication command.

[0032] Additionally, the communication unit (110) incorporates a Quality of Service (QoS)-based transmission control algorithm to evaluate traffic load, network delay, transmission success rate, etc., in real time during data exchange between nodes, and automatically adjusts the priority of the replication schedule. Through this, the transmission speed is limited for nodes in a high-load state, and replication data is prioritized for distribution to low-load nodes, thereby maximizing the network efficiency of the entire cluster. The communication unit (110) also supports secure communication with an administrator terminal or an external audit system outside the cluster, and safely transmits event logs generated by the audit and log recording unit (136) to an external system.

[0033] The communication unit (110) is configured to support stable and efficient execution of system-wide control commands and data exchanges, such as replication, deletion, policy propagation, and log reporting, by managing data exchange with all nodes in the cluster through an integrated communication layer that includes encryption, authentication, QoS, and fault recovery functions.

[0034] The memory (120) is a central data storage configuration that stores and updates instructions, program code, and management information for each node for the control operation of the cluster management device (100), and provides reference data for operations performed by each module (Pin orchestrator, policy management unit, replication consistency management unit, etc.) of the processor (130) to be described later. The memory (120) can be logically divided into a permanent storage area and a high-speed access buffer, and the permanent storage area stores node-specific lifecycle information, data distribution information, replication policy, region policy, retention / destruction policy, and encryption / security key management information. The high-speed access buffer stores temporary operation data, replication status table, and event queue information referenced in real time by the Pin orchestrator (131) and the replication consistency management unit (133) in a cache format. The node-specific lifecycle information stored in memory (120) consists of status variables such as the registration time, operating status, storage capacity, replication history, failure history, and removal time of each node, and is referenced by the Pin orchestrator (131) when determining the addition, deletion, or deactivation of nodes. Additionally, the data distribution information consists of a metadata table containing the unique CID of each data object, replica location, region, replication factor, access frequency, and last modified time, and is used as key standard data when the replication consistency management unit (133) evaluates the replication status.

[0035] The memory (120) stores policy tables periodically delivered from the policy management unit (132) in a cache format, and verifies policy consistency by comparing the version difference between the existing policy and the new policy when changes to replication policies, region policies, recovery policies, etc. occur. At this time, policy updates are performed using a transaction-based atomic update method to ensure that no policy conflicts occur during the replication and data reorganization process. Additionally, the memory (120) includes an event log buffer and a state snapshot area to temporarily store logs of major events (replication start, deletion completion, failure detection, etc.) occurring within the cluster, and synchronizes them with the audit and log record unit (136) at regular intervals.

[0036] The state snapshot area periodically stores the cluster's current data distribution, policy status, and node connection structure, thereby providing a reference point that can be referenced during system recovery or rebalancing. In addition, the memory (120) adds a hash-based signature to each policy and metadata record to verify data integrity and shares the integrity verification results with the replication consistency management unit (133). Through this verification procedure, tampering with or falsification of policy and management data stored in the memory (120) can be prevented. The memory (120) is not merely a simple data storage device, but a policy and metadata integrated management storage that comprehensively manages and verifies node status, replication policies, data distribution, and lifecycle information, and is configured to support the reliable execution of all control and decision-making operations of the cluster management device.

[0037] The processor (130) can perform technical features according to embodiments of the present disclosure to be described below by executing at least one instruction stored in memory (120). The processor (130) may be composed of at least one core and may include a processor for data analysis and / or processing, such as a central processing unit of a computer device, a general-purpose graphics processing unit, a tensor processing unit, etc.

[0038] The processor (130) can train a neural network or model designed using machine learning or deep learning methods. To this end, the processor (130) can perform calculations for training the neural network, such as processing input data for training, extracting features from input data, calculating errors, and updating the neural network's weights using backpropagation. Additionally, the processor (130) can perform inference for a specific purpose using a model implemented in an artificial neural network manner.

[0039] The processor (130) includes a Pin orchestrator (131), a policy management unit (132), a replication consistency management unit (133), a data lifecycle management unit (134), a recovery scheduling unit (135), and an audit and log recording unit (136).

[0040] The Pin orchestrator (131) is configured to centrally and integrally control data replication, deletion, distribution reorganization, and integrity management of the entire cluster. It collects and analyzes status information of each node (operation status, available capacity, network latency, I / O load, error logs, etc.) in real time, and controls the automatic execution of replication, deletion, and reorganization procedures by referring to the replication policy and region policy provided by the policy management unit (132) so that the replication factor (RF) of the entire cluster satisfies the policy criteria.

[0041] The Pin orchestrator (131) periodically receives metadata such as the CID, storage location, number of replicas, and load rate per node for each data from the replication consistency management unit (133) and quantitatively evaluates the degree of imbalance in the current cluster's data distribution. If the evaluation result indicates that the number of replicas for a specific node or region is less than the replication factor specified in the policy, the Pin orchestrator (131) determines the data to be replicated, automatically selects a replication node among normal nodes by comprehensively considering storage capacity, network latency, region priority, failure history, etc., and then performs a dynamic self-healing replication procedure by issuing a replication command to the selected node.

[0042] When a new node is added to the cluster, the Pin orchestrator (131) receives current data distribution and node status information from the replication consistency management unit (133), and then, by referring to the replication policy (replication factor, region policy, data importance criteria, etc.) stored in the policy management unit (132), calculates a list of data to be replicated to the new node, calculates the incremental replication amount based on the CID hash value and reference count of the data, and instructs the incremental rebalancing procedure for the new node to be performed based on the calculation result, and the incremental rebalancing procedure is performed asynchronously in a background thread in parallel with service traffic.

[0043] During the incremental rebalancing process, the entire data is not redistributed; instead, incremental replication is performed only on data that falls below the policy replication factor, enabling uninterrupted data rebalancing without service interruption.

[0044] Additionally, when a deletion request or failure event of a specific node is detected, the Pin orchestrator (131) receives a list of data and reference relationship information stored in the node from the replication consistency management unit (133), and automatically identifies the exclusive data piece that the node holds exclusively.

[0045] Afterwards, a node to be replicated is selected from among the nodes currently in normal operation according to the region policy and available resource criteria of the policy management department (132), and an alternative replication procedure is performed.

[0046] The data evacuation procedure is scheduled to replicate sequentially starting with high-priority data (e.g., CIDs with high importance ratings), and after replication is complete, hash verification is performed on a CID-by-CID basis to confirm the integrity of the data transmission.

[0047] When it is verified that all replications satisfy the replication factor of the policy criteria, the Pin orchestrator (131) transmits a replication completion signal to the replication consistency management unit (133) and then issues a node removal command to the node to be deleted to safely separate the node from the cluster.

[0048] The process is automatically recorded in the audit and log book (136), and the execution time, target data, and result status of the replication, deletion, and rebalancing procedures are stored in an auditable format. As a result, the Pin orchestrator (131) integrates the entire process from maintaining policy-based replication factors, automatically readjusting data distribution, performing service-free replication, verifying integrity, and generating audit logs according to the addition, deletion, and failure events of nodes within the cluster, thereby realizing a highly reliable and self-healing distributed file storage environment.

[0049] The policy management unit (132) stores and manages all policy information related to cluster operation, such as replication policies, region policies, data lifecycle policies, and encryption policies, which are defined by the administrator or automatically generated by the system, and controls each configuration module centrally to maintain policy consistency and operate based on this.

[0050] The policy management department (132) manages policy tables in a multi-layered structure, with global policies defined at the top and region-level policies, tenant-level policies, and file-level policies stored hierarchically connected below them.

[0051] The hierarchical policy structure allows different replication factors and retention policies to be applied to the same data based on importance, usage frequency, retention period, access rights, and security level.

[0052] The policy management unit (132) has a policy decision engine built in, and whenever a policy request is received from the Pin orchestrator (131) or the replication consistency management unit (133), it compares and evaluates the current cluster status information (node ​​load, network bandwidth, replica distribution, region occupancy rate, etc.) with the reference values ​​of the policy table to calculate the replication factor, replication priority, region redistribution rule, and data movement allowance range.

[0053] The calculated policy decision results are transmitted to the Pin orchestrator (131) and used as criteria for performing replication and readjustment procedures. The policy management unit (132) works closely with the replication consistency management unit (133) to periodically verify whether the actual number of replicas and region distribution of each data match the policy criteria. At this time, the policy management unit (132) receives replication status information (number of replicas per CID, storage location per node, access frequency, etc.) reported by the replication consistency management unit (133), automatically detects data objects that are inconsistent with the replication policy, and transmits a policy correction command to the Pin orchestrator (131). Additionally, when an administrator changes a policy or the system automatically updates a policy, the policy management unit (132) controls the system to propagate the changed policy to the entire cluster in real time according to the policy propagation protocol.

[0054] During the propagation process, the policy version management module records the policy version and application time of each node to prevent policy inconsistencies or duplicate reflections and maintain a consistent policy state.

[0055] In particular, when expansion or contraction of a region occurs, the policy management unit (132) analyzes the changed region structure and instructs the Pin orchestrator (131) to perform data movement and replication in an incremental manner according to the new region policy.

[0056] At this time, data movement automatically adjusts the parallelization level and transmission speed considering the load status of the cluster, and the policy management unit (132) can perform a policy application procedure in a step-by-step rollout manner to prevent data inconsistency or replication delay that may occur during policy application. The rollout procedure operates in a feedback structure in which policies are applied sequentially by region or node group, and after the completion of application and verification results of each step are reported to the replication consistency management unit (133), the procedure proceeds to the next step.

[0057] Therefore, data integrity and replication consistency are maintained even during the policy change process, and the impact on service traffic is minimized. The policy management unit (132) transmits all policy change history, application targets, application times, and success status to the audit and log record unit (136) to enable tracking of policy change history and ensures transparency in policy management by linking with an external audit system.

[0058] Accordingly, the policy management unit (132) operates as a policy control hub that performs integrated decision, verification, propagation, application, and audit logic of policies, going beyond simple policy storage functions, thereby ensuring that all replication, deletion, reorganization, and recovery procedures within the cluster are performed consistently according to integrated policy standards.

[0059] The replication consistency management unit (133) is configured to periodically collect and analyze status information, number of replicas, storage space, network connection status, and data access statistics of all nodes in the cluster, and to determine whether the replication status of each data satisfies policy criteria by comparing it with the replication factor (RF) and region policy stored in the policy management unit (132).

[0060] The replication consistency management unit (133) updates the available capacity, connection status, data transmission delay time, I / O load rate, quantity of stored data, and list of replicas per node based on status information packets periodically collected from each node. The collected information is stored in an internal replication status table, which includes metadata such as the unique CID of each data object, replica location, region, replication factor, access frequency, and last update time.

[0061] The replication consistency management unit (133) periodically refers to the replication policy table received from the policy management unit (132) to compare and evaluate the actual number of replicas of each data object with the policy standard replication factor. If, as a result of the comparison, the number of replicas is less than the policy value, the replication consistency management unit (133) generates a data list of the insufficient replicas and generates replication request metadata including the CID, size, importance, region priority, and transmission path bandwidth of each replica, and transmits it to the Pin orchestrator (131). Upon receiving this report, the Pin orchestrator (131) automatically issues a replication command to create the insufficient replicas.

[0062] Conversely, if the number of replicas is greater than the policy value, the replication consistency management unit (133) executes an unnecessary replica detection algorithm to identify unnecessary replicas based on data access frequency, time of recent access, balance of data distribution between regions, storage saturation rate of nodes, etc., and calculates a deletion priority. The deletion priority is calculated by weighting the data importance, the deviation of replication distribution by region, access history weight, etc., and the calculated result is transmitted to the Pin orchestrator (131) so that a policy-based deletion procedure is performed.

[0063] The replication consistency management unit (133) also automatically identifies data objects that are in a state of not meeting the policy replication factor among the data held by the failed node when a node failure or abnormal connection termination is detected, and immediately sends an additional replication trigger signal to the Pin orchestrator (131).

[0064] The Pin orchestrator (131) receives this signal and directs the replication operation to a node among the normal nodes that has sufficient available resources, and the replication consistency management unit (133) monitors the progress and completion status of the replication process in real time. When replication is completed, the replication consistency management unit (133) performs CID-based hash validation on each data to verify transmission integrity, and if it is determined that the replication factor satisfying the policy value has been restored, it marks the data as a Consistent State. At this time, the verification results and the history of the replication state change are transmitted to the audit and log record unit (136) and stored as a replication consistency verification log. In this way, the replication consistency management unit (133) monitors the replication status of the entire cluster in real time, autonomously detects replication factor discrepancies caused by replication imbalance or node failure, and controls the continuous maintenance of replication consistency that meets policy standards by automatically performing replication correction and self-healing replication procedures through integration with the Pin orchestrator (131).

[0065] The data lifecycle management unit (134) is configured to comprehensively manage the entire lifecycle of data, from creation, preservation, modification, archiving, and deletion, by applying retention policies, destruction policies, and version control policies defined at the file, bucket, or tenant level to data objects stored within the cluster.

[0066] The data lifecycle management unit (134) periodically receives a lifecycle policy table from the policy management unit (132), evaluates whether it matches policy conditions based on the creation time, last access time, number of references, number of versions, and associated tenant information of each data object, and controls the automatic execution of destruction procedures for data whose defined retention period has expired or whose reference relationship has been released. The data lifecycle management unit (134) internally has a lifecycle state transition engine and classifies and transitions each data object to one of the following states.

[0067] ① Active, ② Archived, ③ Expired, ④ Deletion Pending, ⑤ Deleted.

[0068] The lifecycle state transition engine automatically performs state transitions based on data access frequency, change history, policy expiration date, etc., and each transition event is notified to the Pin orchestrator (131) to synchronize with the replication state within the cluster.

[0069] When a request for withdrawal from a tenant is detected, the data lifecycle management unit (134) identifies a list of CIDs of all data associated with the tenant from the metadata base, analyzes the reference relationships and shared links for each CID, and selects non-referenced data blocks that are not referenced by other tenants or system processes.

[0070] Afterwards, a Global Unpin Command is sent to all nodes in the cluster through the Pin orchestrator (131), and the Pin status of the corresponding data block is released from the local cache and storage space of each node.

[0071] After the PIN is released, the data lifecycle management unit (134) performs a policy-based unnecessary data removal procedure.

[0072] In the policy-based unnecessary data removal procedure, deletion priorities are determined by comprehensively considering the destruction policy, retention period, access logs, version history, and encryption key retention period of each data, and deletion is performed sequentially, starting with data whose retention period has expired or whose reference relationships have been released.

[0073] The deletion process is performed based on parallel threads, and after deleting each data block, hash validation at the CID level is performed to verify deletion integrity.

[0074] After data deletion is completed, the data lifecycle management unit (134) generates a deletion history log including the CID of the deleted data, the time of deletion, the entity responsible for deletion, the policy basis, and the verification result, and transmits it to the audit and log record unit (136) to record it in the log database. In addition, a deletion result report is transmitted to an external audit system or an administrator console to verify the completeness of the data deletion and compliance with regulations.

[0075] If an error occurs during a deletion operation or if deletion is delayed on some nodes, the data lifecycle management unit (134) reports the error status to the Pin orchestrator (131) and the replication consistency management unit (133), and automatically resumes the retry scheduling or forced deletion procedure for the delayed nodes.

[0076] The complete deletion of data is guaranteed through retry scheduling or forced deletion execution procedures for delayed nodes, and a state consistent with the data retention policy of the entire cluster is maintained. The data lifecycle management unit (134) operates as a Lifecycle Automation Engine that automatically controls and verifies the entire process from the time of data creation to deletion and log recording based on policies, thereby ensuring transparency, integrity, and legal compliance in data management.

[0077] The recovery scheduling unit (135) is configured to restore data replication consistency and integrity of the entire cluster by performing selective recovery and recovery priority setting at the file or data block level when a node failure, network disconnection, storage device error, or data corruption (such as CID hash mismatch) is detected within the cluster.

[0078] The recovery scheduling unit (135) analyzes the replication status report periodically received from the replication consistency management unit (133) and the failure event log received from the Pin orchestrator (131) to extract a list of CIDs of data objects that need to be recovered.

[0079] The recovery scheduling unit (135) determines whether recovery is necessary and the scope of recovery by comprehensively analyzing the CID hash value, affiliated region, replication factor, storage location per node, and integrity verification results of each data. When a recovery target is determined, the recovery scheduling unit (135) operates an internal recovery priority determination engine to quantify indicators such as data importance, access frequency, service impact, availability of resources per region, and policy replication priority. These indicators are calculated using a weighted scoring method, and recovery targets are scheduled to be processed sequentially or in parallel according to the calculated recovery priority score.

[0080] The recovery scheduling unit (135) automatically searches for the original location and alternative replication path of the data to be recovered using a CID-based addressing system during the recovery process and optimizes the data transmission path between normal nodes. At this time, data replication operations between recovery performing nodes are parallelized by considering transmission bandwidth, latency between nodes, and connection costs between regions, and an adaptive throttling algorithm is applied to prevent resource utilization from rising excessively.

[0081] Additionally, the recovery scheduling unit (135) receives real-time status information, such as recovery progress, data transmission success rate, and CID hash verification result, from the replication consistency management unit (133) to monitor the recovery progress status, and if an error occurs during recovery, it immediately transmits a recovery retry command or an alternative path switching command to the Pin orchestrator (131).

[0082] After recovery is complete, the recovery scheduling unit (135) generates a recovery result report including the CID of each recovered data, the recovery source and target nodes, the recovery time, and the verification result, transmits it to the audit and log record unit (136), and transmits a recovery completion signal to the policy management unit (132) to perform consistency updates of the replication policy.

[0083] In this way, the recovery scheduling unit (135) operates in a configuration that maximizes the reliability and availability of the system by automating the entire process from calculating policy-based recovery priority, resource optimization, parallel recovery control, CID integrity verification, and recovery log recording in the event of a failure or data corruption, thereby minimizing the recovery time objective (RTO) and recovery point objective of the cluster.

[0084] Afterwards, the Pin orchestrator (131) issues a replication command according to the recovery plan of the recovery scheduling unit (135), and the replication consistency management unit (133) verifies the integrity of the recovered data, thereby minimizing the recovery time target of the cluster.

[0085] The audit and log recorder (136) is configured to collect all events, such as replication, deletion, recovery, and policy changes, in real time and record them in a log database in chronological order. Log data is stored in an immutable structure (Append-Only Ledger) that is cryptographically hash-signed to prevent tampering, and is periodically synchronized with an external audit system to ensure transparency in data governance. Additionally, the audit and log recorder (136) can automatically detect abnormal events (e.g., unauthorized deletion, replication failure, policy inconsistency) and send a warning to an administrator terminal.

[0086] FIG. 2 is a flowchart illustrating a cluster management method for a distributed file system according to an embodiment of the present invention, FIG. 3 is a detailed flowchart of process S710 shown in FIG. 2, FIG. 4 is a detailed flowchart of process S720 shown in FIG. 2, and FIG. 5 is a detailed flowchart of process S730 shown in FIG. 2.

[0087] First, referring to FIG. 2, a cluster management method (S700) for a distributed file system according to an embodiment of the present invention can be implemented through the device shown in FIG. 1.

[0088] More specifically, S700 proceeds as a series of processes in which, when a new node is added to the cluster, the new node calculates a list of data to be replicated based on the current cluster's data distribution and replication policy, performs data reorganization (S710), and when a specific node is deleted, performs a data evacuation procedure to replicate data fragments stored in the node to another node, then separates the specific node from the cluster (S720), applies a dynamic replication policy according to the importance or frequency of use of the data, sets a replication factor, and monitors the cluster status to automatically adjust the actual number of replicas to match the policy (S730).

[0089] Referring to FIG. 3, process S710 is a process of calculating a data list and performing incremental reallocation when a new node is added. The replication consistency management unit collects the cluster's current data distribution, number of replicas, available capacity per node, network latency, and I / O load rate, and updates the metadata base (S711). Then, the policy management unit provides reallocation-related policy values, such as replication factor, region policy, data importance criteria, transmission concurrency upper limit, and rate limit value (S712). Subsequently, the Pin orchestrator extracts candidate data to be placed on the new node, calculates an incremental placement score based on each data's identifier, size, importance, reference count, and current replica location (S713), and creates a plan to transfer items with the highest scores to the new node, while applying transmission concurrency and rate limits to minimize interference with service traffic (S714). Next, incremental replication is performed in a background thread, and metadata is updated after integrity verification upon completion of each placement (S715). At this time, full redistribution is not performed, and only the insufficient sections are reinforced. Finally, the process may include a series of steps such as checking whether the target replication factor has been reached, storing the transmission volume, time taken, and failure details in the audit and logbook, and registering unreached items in the retry queue (S716).

[0090] Referring to Fig. 4, process S720 is a node separation process that performs a data evacuation procedure upon deletion of a specific node. When a request to delete a specific node is received due to a management directive or a failure determination, the replication consistency management unit queries the list of data held by the node and reference relationships (S721), identifies exclusive data fragments existing only on that node and data objects that do not meet the replication factor, and assigns priorities (S722). A set of replacement nodes is calculated (S723) considering storage free space, latency, and load rate by reflecting the region policy and resource status of the policy management unit, and the Pin orchestrator directs parallel replication according to the evacuation priority and performs adaptive rate limiting and concurrency control to prevent resource overload during transmission (S724). Once process S724 is completed, the integrity of each data is verified, and whether the target replication factor is satisfied is repeatedly checked. It includes a series of processes such as immediately retrying unmet items (S725), controlling the node to be detached from the cluster when it is confirmed that the evacuation of all target data is complete, cleaning up node registration information and routing information (S726), recording the evacuation target, replacement node, verification result, and detachment time in the audit log, and synchronizing with an external audit system (S727).

[0091] Referring to FIG. 5, the S730 process involves applying a dynamic replication policy, setting replication factors, and automatic matching. The policy management unit activates a dynamic replication policy and replication factor range that reflects data importance, access frequency, and retention period (S731), and the replication consistency management unit determines a shortage, excess, or matched state by comparing the actual number of replicas with the policy value (S732). Subsequently, the Pin orchestrator instructs additional replication for shortage items and removal of unnecessary replicas for excess items, while also considering region balance and network conditions (S733). Afterward, it adjusts the correction cycle, concurrency, and target list in real-time according to policy changes, traffic fluctuations, and changes in node resources (S734).

[0092] Afterward, the process may include a series of steps such as verifying whether the replication factor is satisfied and the integrity again after correction, marking the data as normal consistency state if satisfied (S735), saving the correction command, target data, execution time, and results in the audit and logbook, and reporting to an external audit system if necessary (S736).

[0093] Therefore, the present invention provides the advantage of uninterrupted operation by automatically performing replication and data reorganization without affecting service traffic, even when nodes are added or deleted. The present invention provides the advantage of simultaneously securing storage efficiency and stability by dynamically controlling replication factors according to data importance and usage frequency. The present invention provides the advantage of complying with personal data protection regulations, such as GDPR, by automatically performing complete data deletion upon tenant withdrawal. The present invention provides the advantage of minimizing the Recovery Time Objective (RTO) by supporting file-unit selective recovery using a CID-based addressing system. The present invention provides the advantage of simplified operation, as the system is controlled to automatically perform self-healing replication and deletion simply by changing policies by an administrator. The present invention provides the advantage of enhanced traceability for external audits and governance responsiveness by logging and storing all histories, such as replication, deletion, and policy changes, in chronological order.

[0094] Meanwhile, the methods according to the various embodiments of the present invention described above may be implemented in the form of an application or software program that can be installed on an existing electronic device. In addition, all or part of the method may be composed of multiple software function modules and implemented on an operating system (OS). Alternatively, each step may be composed of a single software function module, or each step may be combined to form a single software function module and implemented on an operating system. Therefore, even if all of the embodiments of the present disclosure are not implemented as a single software function module, if multiple software function modules implement each step of the present disclosure and multiple software function modules are implemented on a single operating system, it can be understood that the method of the present disclosure has been implemented.

[0095] In addition, the methods according to the various embodiments of the present invention described above can be implemented solely through software upgrades or hardware upgrades of existing electronic devices. Furthermore, the various embodiments of the present invention described above can also be performed through an embedded server equipped in an electronic device or an external server of the electronic device.

[0096] Meanwhile, according to one embodiment of the present invention, the various embodiments described above may be implemented as software comprising instructions stored on a computer-readable recording medium using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented as the processor itself. According to the software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Meanwhile, a computer or similar device may include a device according to the disclosed embodiments, which is capable of calling instructions stored from a storage medium and operating according to the called instructions. When said instructions are executed by a processor, the processor may perform the function corresponding to said instructions directly or using other components under the control of said processor. The instructions may include code generated or executed by a compiler or an interpreter. A machine-readable recording medium may be provided in the form of a non-transitory computer-readable recording medium. Here, "non-transitory" simply means that the storage medium does not contain a signal and is tangible; it does not distinguish whether data is stored semi-permanently or temporarily on the storage medium. In this context, a non-transitory computer-readable medium refers to a medium that stores data semi-permanently and is readable by a machine, rather than a medium that stores data for a brief moment, such as registers, caches, or memory.Specific examples of non-transient computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, ROMs, etc.

[0097] As described above, exemplary embodiments have been disclosed in the drawings and specification. Although specific terms have been used to describe the embodiments in this specification, they are used only for the purpose of explaining the technical concept of this disclosure and are not intended to limit the meaning or the scope of this disclosure as defined in the claims. Therefore, those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of this disclosure should be determined by the technical concept of the appended claims. Explanation of the symbols

[0098] 100: Cluster management device for distributed file storage systems 110: Communications Department 120: Memory 130: Processor 131: Pin Orchestrator 132: Policy Management Department 133: Replication Consistency Management Department 134: Data Lifecycle Management Department 135: Recovery Scheduling Section 136: Audit and Log Book

Claims

Claim 1 A cluster management device for a distributed file storage system (IPFS), comprising: a communication unit that communicates with a plurality of nodes within a cluster; a memory that stores management information including at least one instruction and a node-specific lifecycle, data distribution, and replication policy; and a processor that performs operations according to the instruction, wherein when a new node is added to the cluster, the processor calculates a list of data that the new node must replicate based on the current cluster's data distribution and replication policy and performs data reorganization, and when a specific node is deleted, controls the specific node to be separated from the cluster after performing a data evacuation procedure to replicate data fragments stored in the node to another node, and wherein the processor applies a dynamic replication policy according to the importance or frequency of use of the data, sets a replication factor, and monitors the cluster status to automatically adjust the actual number of replicas so that it matches the policy. Claim 2 In claim 1, the processor is a cluster management device that centrally manages the lifecycle of all nodes through a Pin orchestrator. Claim 3 A cluster management device according to claim 1, wherein the processor performs automatic data distribution by issuing a data rebalancing command considering the load status of the cluster when the new node is added. Claim 4 A cluster management device according to claim 1, wherein the processor identifies data fragments stored only in the corresponding node when the specific node is deleted, and delays deletion until replication to another normal node is completed. Claim 5 In paragraph 4, the above processor is a cluster management device that executes node separation only after confirming the completion of replication to prevent data loss during the data evacuation procedure. Claim 6 delete Claim 7 In claim 1, the processor is a cluster management device that supports setting different replication factors for specific data on a file or folder basis. Claim 8 In claim 1, the processor is a cluster management device that performs additional replication when the number of replicas is less than the policy when a node failure occurs, and deletes unnecessary replicas when the number is more than the policy. Claim 9 In paragraph 1, the cluster management device controls the processor to operate without service interruption during data reorganization or evacuation procedures. Claim 10 A cluster management method for a distributed file storage system, comprising: a step of, when a new node is added to a cluster in a processor, calculating a list of data to be replicated by the new node based on the data distribution and replication policy of the current cluster and performing data reorganization; a step of, when a specific node is deleted in the processor, performing a data evacuation procedure to replicate data fragments stored in the node to another node and then separating the specific node from the cluster; and a step of, in the processor, applying a dynamic replication policy according to the importance or frequency of use of the data, setting a replication factor, and monitoring the cluster status to automatically adjust the actual number of replicas so that it matches the policy.

Images