Virtual data replication to support garbage collection in distributed file systems

By introducing virtual data replication and logical address mapping in the distributed file system, the problem of thread blocking during the garbage collection process is solved, computing performance and storage service reliability are improved, and costs are reduced.

CN116490847BActive Publication Date: 2024-11-15ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080106598.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-05
Publication Date
2024-11-15
Estimated Expiration
2040-11-05

AI Technical Summary

Technical Problem

In distributed file systems, thread blocking workloads caused by garbage collection processes can affect computing performance, especially in cloud computing environments, resulting in reduced service quality and increased costs.

Method used

By introducing virtual data replication methods in distributed file systems, remapping is done using logical address mappers to avoid actual data replication, reduce disk-level workloads across nodes, and implement virtual replication operations to support preemptive garbage collection strategies.

Benefits of technology

Reduces the need for data operation workload and compression and defragmentation, reduces thread blocking, improves the performance and reliability of storage services, and avoids the deterioration of computing performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116490847B_ABST
    Figure CN116490847B_ABST
Patent Text Reader

Abstract

The method and system implement a virtual copy operation that improves support for preemptive garbage collection strategies at a distributed file system. The garbage collection process preemptively performs a copy of data in a log-structured file system. In order to prevent thread-blocking write operations from being distributed across nodes of a distributed file system node cluster, thereby avoiding degradation of computing performance, a virtual copy operation is provided that locates each block node storing a block to be virtually copied based on master node metadata, and calls a remapping API of a logical address mapper, which is provided by a local file system of the block node. The logical address mapper of each block node performs a remapping of a disk address from a source logical block address to a destination logical block address without relocating data from one disk address to another. The results of these remappings can be stored as metadata at the master node, replacing previously mapped metadata.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Data storage has increasingly entered the realm of cloud computing, where hosting file systems on networked distributed servers greatly improves the availability and reliability of remotely stored files and enables data workloads to be served by the same distributed computing resources, which can be scaled to meet the needs of large-scale computing applications and projects. Therefore, it is desirable for a distributed file system ("DFS") to establish access to files in a manner that is common to and integrated with local file systems. The Portable Operating System Interface ("POSIX") application programming interface ("API"), which has long been established as a standard for system calls, has been increasingly supported in various DFS implementations to establish compatibility with local file systems. In addition, DFS implementations also provide a web service API that can be accessed through a network connection.

[0002] In many competing implementations of DFS from vendors such as cloud computing and cloud service providers, the APIs typically provide abstractions that hide the underlying hardware design and architecture and hide the management of files and resources from end users of distributed storage and distributed computing resources. For example, calls to the DFS APIs from end-user computing devices to read and write remotely stored files may result in copying and moving data of these stored files between different distributed nodes in a data center or cloud computing system without making such copying and moving transparent to the end user.

[0003] In order to maintain an abstraction that hides the underlying architecture and resource management from end users, it is increasingly important to execute the processes triggered by DFS API calls in an efficient manner. Therefore, while the processes may have an intensive role in managing distributed storage resources and distributed computing resources, the computational workload generated by these processes should not be so intensive as to interfere with user-level processes and reduce the efficiency of these user-level processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The detailed description is explained with reference to the accompanying drawings. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.

[0005] Figure 1 An architectural diagram of a computing system according to an example embodiment of the present disclosure is shown.

[0006] Figure 2 An architectural diagram of a distributed file system node cluster according to an example embodiment of the present disclosure is shown.

[0007] Figure 3 A flowchart of a garbage collection method that can call a virtual data copy method according to an example embodiment of the present disclosure is shown.

[0008] Figure 4A and Figure 4B A flowchart of a virtual data replication method according to an exemplary embodiment of the present disclosure is shown.

[0009] Figure 5 A flowchart of a remapping method performed at a logical address mapper of a managed storage of a chunk node according to an example embodiment of the present disclosure is shown.

[0010] Figure 6 Shows Figure 3 , Figure 4A , Figure 4B and Figure 5 A summary of the method, as in reference Figure 2 shown.

[0011] Fig. 7A and Figure 7B An example distributed file system for implementing the processes and methods described herein for virtual data replication is shown.

[0012] Fig. 8A and Figure 8B An example block node is shown for implementing the processes and methods described herein for virtual data replication.

[0013] 9A to 9D An example multi-tier system for implementing the processes and methods described herein for virtual data replication is shown. DETAILED DESCRIPTION

[0014] The systems and methods discussed herein are directed to implementing a distributed file system, and more particularly, to implementing a distributed file system in which virtual data replication at block nodes of a node cluster enhances the performance of a garbage collection process implemented by a local file system.

[0015] Figure 1 An architectural diagram of a computing system 100 according to an example embodiment of the present disclosure is shown. The computing system 100 may be a cloud computing system that may provide a collection of servers that host storage resources and computing resources to provide a distributed file system, distributed computing and parallel computing, improved availability of physical or virtual storage resources and computer resources, and similar benefits.

[0016] The computing system 100 may be implemented through a cloud network 102 of physical or virtual server nodes 104 connected through physical or virtual network connections. In addition, the network 102 may terminate at a physical or virtual edge node 106 located at the physical edge and / or logical edge of the cloud network 102. The edge node 106 may be connected to any number of end devices 108.

[0017] A distributed file system (“DFS”) may be implemented on the node clusters 112 of the cloud network 102. Some number of node clusters 112 may be hosted at one or more data centers 114. Each data center 114 may include some number of server nodes 104, which are logically organized into some number of node clusters 112, wherein the server nodes 104 may be nodes of the cloud network 102 or may be nodes that communicate with the cloud network 102 via a network connection.

[0018] Within the DFS node cluster 112, the server nodes 104 in the DFS node cluster 112 may include a master node 116 and a plurality of block nodes 118. The DFS hosted at the DFS node cluster 112 may configure the master node 116 and each block node 118 to communicate with any number of terminal devices 108 and any number of server nodes 104 of the cloud network 102, respectively, through any number of network connections, according to a file system communication protocol that implements one or more application programming interfaces ("APIs") that provide file operation calls. The file operation calls include, for example, API calls that cause each block node 118 to create a file, delete a file, read from a file, write a file, rename a file, move a file from one location in a file system hierarchy to another location in the file system hierarchy, and the like.

[0019] Physical storage and / or virtual storage (“hosted storage 120”) may be hosted at each DFS node cluster 112. For example, physical storage and / or virtual storage may be hosted at one or more block nodes 118 of a DFS node cluster 112. Data may be stored as logical blocks of predetermined size, which may each be individually referred to as a “block.” DFS may configure the block nodes 118 together to create multiple copies of each individual data block, wherein the individual copies may be distributedly stored on multiple block nodes 118, providing redundancy and increasing the availability of all data blocks stored at the DFS node cluster 112.

[0020] The DFS may also configure each block node 118 to perform memory operations and disk operations on the managed storage 120. For example, the block node 118 may be configured to: read data stored on the managed storage 120, write data to the managed storage 120, copy data from one address on the managed storage 120 to another address on the managed storage 120 or copy data from one logical block on the managed storage 120 to another logical block on the managed storage 120, move data from one address on the managed storage 120 to another address on the managed storage 120 or move data from one logical block on the managed storage 120 to another logical block on the managed storage 120, etc. As will be described later, these operations, such as disk operations, may be performed by the local file system.

[0021] DFS may also configure master node 116 to store metadata regarding the location of each data block in block node 118. The metadata may provide a mapping between: a file according to the file system hierarchy (as defined by DFS); and each data block that makes up the file, including a copy of each data block as located in block node 118.

[0022] The computing system 100 may also implement one or more storage services. Using the managed storage 120 as the storage resource underlying the storage service, the storage service may be hosted at any number of node clusters of the computing system 100 and at any number of server nodes in the computing system 100. The storage service may be hosted at least in part at server nodes 104 that are not part of the DFS node cluster 112, and the storage service may be additionally hosted at least in part at some server nodes 104 of the DFS node cluster 112. For example, the one or more storage services may include: various implementations of storage architectures, such as, for example, block device services, object storage, distributed file systems, etc.; and any such appropriate data storage implementation known to those skilled in the art that benefits from managed storage distributed across multiple server nodes. As will be described subsequently, the one or more storage services may be provided as managed services to clients of the DFS node cluster 112.

[0023] DFS may also implement a garbage collection ("GC") coordinator process at the DFS node cluster 112. As will be described subsequently, the GC coordinator process may provide GC services for the DFS and one or more storage services implemented in the DFS to improve the efficiency of the DFS and the one or more storage services' read and write performance to the underlying managed storage. The GC coordinator process may run on one or more coordination server nodes of the DFS node cluster 112, such as, for example, at least one master node 116, or the GC coordinator process may generally run in any suitable manner such that the one or more coordination server nodes running the GC coordinator process may communicate with any number of other server nodes of the DFS node cluster 112, each of which runs a GC worker process. As will be described subsequently, the GC coordinator process may be provided as a managed service to clients of the DFS node cluster 112.

[0024] The file system communication protocol described herein may implement an API such as a Portable Operating System Interface (“POSIX”) API, a File System in Userspace (“FUSE”) API, a Network File System (“NFS”) API, a Representational State Transfer (“REST”) API, or the like.

[0025] The hosted storage 120 may include various forms of computer-readable storage media, which may include volatile memory (e.g., random access memory ("RAM")) and / or non-volatile memory (e.g., read-only memory ("ROM"), flash memory, etc.). The computer-readable storage media may also include additional removable storage and / or non-removable storage, which may include, but is not limited to, flash memory, magnetic storage, optical storage, and / or tape storage, which may provide non-volatile storage of computer-readable instructions, data structures, program modules, etc.

[0026] Non-temporary computer-readable storage media are examples of computer-readable media. Computer-readable media include at least two types of computer-readable media, namely computer-readable storage media and communication media. Computer-readable storage media include volatile media and non-volatile media, removable media and non-removable media, which are implemented in any process or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media include, but are not limited to, phase change memory ("PRAM"), static random access memory ("SRAM"), dynamic random access memory ("DRAM"), other types of random access memory ("RAM"), read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), flash memory or other memory technology, compact disc read-only memory ("CD-ROM"), digital versatile disk ("DVD") or other optical storage, cassette, tape, disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media can embody computer-readable instructions, data structures, program modules or other data in a modulated data signal or other transmission mechanism such as a carrier wave. As defined herein, computer-readable storage media do not include communications media.

[0027] The computing system 100 may include any number of DFS node clusters 112. Each DFS node cluster 112 may implement a separate DFS that provides services for different data storage workloads. The server nodes 104 of each DFS node cluster 112 may also host services responsible for different computing workloads, which may generate data to be stored at the block nodes 118 of the file system node cluster 112. Without limitation, the hosted service may include one or more storage services and the GC coordinator process described above. The hosted service may be accessed by a number of terminal devices 108 via a network connection, and these terminal devices 108 may act as clients of the hosted service when the computing resources of the server nodes 104 are used to perform locally generated computing tasks. When DFS is used as an extension or entirety of the file system for computing tasks performed by the server nodes 104, these server nodes 104 may in turn act as clients of the DFS node cluster 112.

[0028] Through the implementation of the computing system 100 as described above, a computing task including some number of computer executable instructions can be locally originated from the terminal device 108, and the terminal device 108 can make a managed service API call to the server nodes 104 of the DFS node cluster 112, so that these server nodes 104 execute the computer executable instructions constituting the computing task. The execution of the computer executable instructions can configure the server node 104 to make a file operation API call to the DFS implemented on the master node 116 and the block node 118. The data generated by executing the computer executable instructions constituting the computing task can then be stored at the block node 118 in the form of blocks and in the form of some number of copies.

[0029] Figure 2 An architectural diagram of a DFS node cluster 112 according to an example embodiment of the present disclosure is shown. Shown are master nodes 116 and some number of block nodes 118. Each block node 118 stores some number of blocks 202, wherein any block stored on an individual block node 118 may be a copy of at least some number of other blocks stored on other individual block nodes 118. The master node 116 stores metadata 204 that provides a mapping between: a file according to a file system hierarchy (as defined by DFS); and each data block that makes up the file, the file including a copy of each data block as located in a block node 118.

[0030] The client 206 of the DFS node cluster 112 may be any computing system that performs the following operations: the computing system performs computing tasks and utilizes the block nodes 118 of the DFS node cluster 112 as an extension of the local file system, utilizes the block nodes 118 of the DFS node cluster 112 instead of the local file system, utilizes the block nodes 118 of the DFS node cluster 112 with the ability to virtualize the local file system, etc. For example, the client 206 may be a server node 104 of the cloud network 102 in addition to the server nodes of the DFS node cluster 112, or may be a server node 104 of the DFS node cluster 112 including the master node 116 and the block node 118.

[0031] According to an example embodiment of the present disclosure, the DFS 212 implemented at the DFS node cluster 112 may logically organize files as a log-structured file system, wherein each modification of data stored in the DFS may be stored as an append to the head of a sequential data structure such as a log. Thus, a call to the DFS 212 to perform any type of logical file operation in which data is modified may cause the DFS file system to write an append to the head of such a sequential data structure, the append reflecting the modification made by the file operation.

[0032] In addition, each block node 118 can be configured to perform disk operations, such as reading data stored on the local storage 210 of the block node 118, writing data to the local storage 210, copying data from one address on the local storage 210 to another address on the local storage 210 or copying data from one logical block on the local storage 210 to another logical block on the local storage 210, moving data from one address on the local storage 210 to another address on the local storage 210 or moving data from one logical block on the local storage 210 to another logical block on the local storage 210, etc. Each such disk operation can be performed by calling the local file system 208 of the corresponding block node 118.

[0033] According to an example embodiment of the present disclosure, the local storage 210 of the block node 118 may include a subset of the hosted storage 120 .

[0034] Implementing the DFS file system as a log-structured file system can also cause subsequent appendages to replace previous appendages, making previous appendages obsolete, while reflecting more current versions of data written to the file system. Therefore, the DFS node cluster 112 can also implement a garbage collection ("GC") coordinator process that controls GC worker processes at the DFS node cluster 112. Typically, GC is a background operation that performs copy operations and move operations (as described above) to reclaim storage space by copying valid data to new files, marking old data as invalid, and ultimately reclaiming the space occupied by the invalid data. The GC coordinator process, together with the running GC worker processes, can traverse the sequential data structures of the DFS 212, identify data in obsolete appendages, and reclaim the storage space occupied by obsolete appendages by writing subsequent appendages in the storage space occupied by the obsolete appendages. The GC worker process can typically be executed by a block node according to an example embodiment of the present disclosure as a background process, which has a lower priority than, for example, executing computer executable instructions that constitute a computing task, and performing disk operations. Any and all storage services built on a log-structured file system can rely on the GC process to reclaim storage space by copying valid data to new files, marking old data as invalid, and ultimately reclaiming the space occupied by the invalid data, as described subsequently. These storage services are such as block device services, object storage, distributed file systems, etc., as described above, but not limited to these.

[0035] It should be understood that although the GC worker process running on the server node can be a background process, the GC worker process can be run as a foreground process to perform space reclamation as described above. Therefore, each time the GC worker process reclaims space in the file system of DFS 212, in a phenomenon known as thread blocking, the execution of other instructions by the node running the GC worker process may be suspended until the GC worker process completes the reclamation. Higher utilization of storage space across DFS (including across individual block nodes 118) is typically associated with more frequent reclamation of space by the GC worker process, resulting in more frequent suspension of normal workload nodes to perform GC reads and GC writes. This dedicated GC compute workload that causes thread blocking is a feature of file systems that rely on GC worker processes, such as certain implementations of log-structured file systems.

[0036] According to the implementation of DFS in the example embodiment of the present disclosure, the GC coordinator process may cause the GC worker process to reclaim the space mapped to the data stored on a block node corresponding to the copy of the same data stored on other block nodes. It will be understood by those skilled in the art that the implementation of DFS may enable the GC coordinator process to coordinate multiple block nodes 118 to propagate changes to the stored data at one of the copies of the stored data to each other copy of the same data in the block node, so that the copies of the same data stored in the block nodes of the DFS node cluster 112 are synchronized. In order to understand the example embodiment of the present disclosure, it is not necessary to further describe the synchronization of multiple copies of data in detail herein, except that it should be noted that this synchronization applied to the space recovery coordinated by the GC coordinator process may cause a dedicated GC worker process to calculate the workload, and may cause additional computing workloads for multiple server nodes across the network, such as reading data at one server node and sending messages via the file system communication protocol described above so that the read data is written at some number of other server nodes. This propagation of thread blocking workloads across cloud networks can cause large-scale performance degradation. Therefore, as described above, during normal operation, any and all storage services hosted at server nodes 104 may suffer from performance degradation. Although GC is nominally a background process, in operation, GC may divert a large amount of computing and network resources away from the hosted storage services, especially on node clusters with high capacity utilization. Therefore, for cloud computing and cloud service providers, the quality of service ("QoS") and the cost of providing services may be adversely affected.

[0037] In addition, other data operations performed by the log-structured file system, such as compression, defragmentation, and deduplication of logical blocks of the file system, may also cause the propagation thread to block the workload when propagating synchronization operations across multiple copies of the data in the DFS node cluster 112.

[0038] Therefore, based on a computing system implementing DFS at some number of DFS node clusters as described herein, an example embodiment of the present disclosure provides a method of virtual data replication and synchronization thereof on a DFS node cluster to remap logical blocks at a local file system. Due to the implementation of such a virtual data replication method, any and all storage services as described above hosted at a server node of a DFS node cluster can obtain the following benefits: reduced data operation workload, and reduced compression, defragmentation, and deduplication; the details of these benefits will be described later.

[0039] According to an example embodiment of the present disclosure, the managed storage at each block node of the DFS node cluster can be implemented at least in part using any suitable storage device that can be configured to map logical block addresses to disk addresses (as will be described later), such as flash memory. The flash memory incorporated at the server node can provide a logical block addressing ("LBA") remapping API call through the local file system.

[0040] According to example embodiments of the present disclosure, the remapping API call may be implemented in various APIs as described herein, such as a Portable Operating System Interface (“POSIX”) API, a File System in User Space (“FUSE”) API, a Network File System (“NFS”) API, a Representational State Transfer (“REST”) API, etc. Although the remapping API call may be called from computer executable instructions running in a user space of a computing system, instructions called by the remapping API call (as will be described later) may be executed in a kernel space of the computing system.

[0041] The remapping API call may act on a storage controller of the managed storage, which may implement a flash translation layer ("FTL"), a subsystem of a local file system on a block node where the managed storage resides. The FTL may implement a mapping of logical block addresses ("LBAs"), such as logical addresses of blocks in the managed storage, to disk addresses of an underlying storage device, such as flash memory. The mapping may be stored in an index map, which is stored in the memory of one or more storage devices underlying the managed storage. Although the LBAs may be addressed in a logical order decoupled from any particular disk address, each LBA may be mapped to any disk address on the underlying storage device (which may be identified by block, page, and sector identifiers, etc. established for addressing the flash memory device). The FTL may also implement remapping of LBAs to different disk addresses, such as during the normal course of flash file system operation (e.g., during the wear leveling process of a flash memory device known to those skilled in the art, the underlying data corresponding to a particular LBA may need to be relocated to a different disk address as a matter of course).

[0042] More generally, the remapping API may act on a logical address mapper of managed storage. Managed storage may be implemented by a variety of underlying storage devices, not limited to flash memory; in such other cases, the storage device may still implement a mapping of LBAs to disk addresses of the underlying storage device, the disk addresses conforming to any disk addressing system known to those skilled in the art. Such mapping may be implemented in a storage device driver, on storage device firmware, in an integrated circuit in a physical controller circuit of a storage device, and the like. For purposes of the example embodiments of the present disclosure, for the sake of brevity, each such implementation of a mapping of LBAs to disk addresses (including as given above, but not limited to FTL and each other example) may be referred to as a "logical address mapper."

[0043] According to an example embodiment of the present disclosure, the DFS implemented at the DFS node cluster may implement APIs such as a Portable Operating System Interface (“POSIX”) API, a User Space File System (“FUSE”) API, a Network File System (“NFS”) API, a Representational State Transfer (“REST”) API, etc. These APIs face clients of the DFS node cluster, and these APIs may provide abstractions that hide the physical and logical organization of the underlying managed storage from the client, and hide the underlying local file system APIs such as the remapping API from the client.

[0044] Figure 3A flowchart of a garbage collection method 300 that can call a virtual data replication method according to an example embodiment of the present disclosure is shown. According to an example embodiment of the present disclosure, the garbage collection method 300 can be implemented at a DFS hosted at a DFS node cluster. It should be understood that although the garbage collection process runs in the background of the server nodes of the DFS node cluster as described later, the processes of one or more hosted storage services as described above can run in the foreground of any number of these server nodes at the same time. As described above, this foreground execution of the storage service's processes can cause disk operations, and the execution of these disk operations (these disk operations do not need to be described in further detail) can meet the following conditions: for example, GC trigger conditions and replication thresholds. Therefore, the following may be referred to as a virtual replication operation called by a disk operation of the storage service: as described above, the subsequent steps that are executed due to the replication threshold and GC trigger conditions that are met after certain disk operations of the storage service.

[0045] At step 302, a garbage collection ("GC") worker process is triggered to wake up by satisfying a GC trigger condition detected by a GC coordinator process, the GC worker process running on a server node of a distributed file system ("DFS") node cluster.

[0046] As described above, the GC coordinator process can run on one or more coordinating server nodes of the DFS node cluster (for example, on at least one master node), or the GC coordinator process can generally run in any suitable manner such that the one or more coordinating server nodes running the GC coordinator process can communicate with any number of other server nodes of the DFS node cluster, each of the other server nodes in the DFS node cluster 112 running a GC worker process.

[0047] The GC worker process can also run on one or more server nodes of the DFS node cluster and can run in a thread that runs as a background thread on the server node in a dormant state.

[0048] The satisfaction of a GC trigger condition may be reflected in the composition of data appended to a sequential data structure such as a log as implemented by a log-structured file system. For example, a GC trigger condition may be satisfied once the ratio of stale appendages to non-stale appendages in a sequential data structure exceeds a ratio threshold. Additionally, GC trigger conditions may be based on compositions such as ratios that are measured across most or all server nodes of a DFS node cluster and summed, averaged, normalized, or otherwise aggregated to reflect the overall composition of data at the managed storage of the DFS node cluster. A GC coordinator process may monitor the composition of data across most or all server nodes to determine whether a GC trigger condition is satisfied.

[0049] The GC coordinator process may then send a signal across one or more network connections of the DFS node cluster, where the signal may be, for example, a kernel-level system call that is used to trigger a thread running a GC worker process to wake up.

[0050] In step 304, the GC worker process determines a first data range and a second data range of the sequential data structure.

[0051] At the DFS, the GC working process may traverse a sequential data structure implemented by a DFS file system (e.g., a log-structured file system), and for each continuous data range within the sequential data structure (it should be understood that a continuous data range with respect to a sequential data structure refers to continuous logical block addresses in the sequential data structure, and these continuous logical block addresses are mapped to corresponding disk addresses, and the corresponding disk addresses are not necessarily continuous), determine whether the continuous data range is less than or greater than the virtual copy threshold. Each smaller continuous data range may be determined as a first data range of data to be copied, and each such data range may be recorded in a first list data structure. Each larger continuous data range may be determined as a second data range of data to be virtually copied, and each such data range may be recorded in a second list data structure.

[0052] The virtual copy threshold may be, for example, a size of several kilobytes.

[0053] The GC worker process may perform step 304 under the coordination of a GC coordinator process running on a coordinating server node through one or more signals sent across one or more network connections of the DFS node cluster.

[0054] In step 306 , the GC working process creates a new sequential data structure and copies the data of each first data range to the new sequential data structure.

[0055] The GC worker process creates a new sequential data structure such as a new log to reorganize the non-stale appended data into a compressed, non-fragmented range of disk addresses. In this way, after the copy is complete, the entire previous sequential data structure including the stale appended data can be completely released and recycled.

[0056] However, two types of copy operations occur, including a "virtual copy" operation according to an example embodiment of the present disclosure. It should be understood that, in contrast to an actual copy operation, a virtual copy does not require the execution of data operations such as read operations and write operations. Instead, a virtual copy operation is performed to establish a logical block address ("LBA") for a second data range of data to be virtually copied in a new sequential data structure without locally copying the data to the sequential data structure, and the logical block address is mapped to the disk address where the data to be virtually copied actually resides.

[0057] By invoking a virtual copy instead of a normal copy operation on at least a portion of the new sequential data structure, a GC worker process according to an example embodiment of the present disclosure can avoid a large number of thread blocking workloads on a cloud network hosting a DFS node cluster. The larger the continuous data range, the more likely it is that at least some of the second data ranges involve data to be virtually copied that is not stored locally on other block nodes in the DFS node cluster. Therefore, larger continuous data ranges should be virtually copied instead of normally copying larger continuous data ranges to minimize the performance impact of this statistical possibility.

[0058] The GC worker process may perform step 306 under the coordination of the GC coordinator process running on the coordinating server node through one or more signals sent across one or more network connections of the DFS node cluster.

[0059] At step 308, the GC worker process passes each second data range, the previous sequential data structure, and the new sequential data structure to a virtual copy operation call of the DFS API as parameters.

[0060] Instead of implementing virtual copy operations in the local file system of the block node, virtual copy operations can be implemented at the DFS level, so that virtual copy operations can be performed on an inter-block node basis while hiding the inter-block node disk operations and network communications from the local file system. Figure 4A and Figure 4B The virtual copy operation is further described in detail, and the details of the virtual copy operation can be hidden from the GC worker process.

[0061] The GC worker process may perform step 308 under the coordination of the GC coordinator process running on the coordinating server node through one or more signals sent across one or more network connections of the DFS node cluster.

[0062] In step 310 , the GC working process obtains a logical address corresponding to each second data range based on a virtual copy operation call.

[0063] As a follow-up reference Figure 5 As described, the GC work process can, for example, obtain a logical address based on the destination LBA returned by the logical address mapper. Alternatively, the GC work process can map different logical addresses to the destination LBA returned by the logical address mapper. Alternatively, the GC work process can assign a logical address instead of a destination LBA to correspond to each second data range.

[0064] By receiving this output, the GC worker process can understand the virtual copy operation as completed.

[0065] The GC worker process may transmit the obtained logical address corresponding to each second data range to the GC coordinator process running on the coordinating server node via one or more signals sent across one or more network connections of the DFS node cluster.

[0066] In step 312, the GC coordinator process notifies the master node of the DFS node cluster of the new sequential data structure.

[0067] The GC coordinator process can communicate with the master node according to the file system communication protocol as described above, and the GC coordinator process can call the DFS API call to configure the master node to update the mapping between the files according to the file system hierarchy and each data block that constitutes the file, so that the file is mapped to the blocks of the new sequential data structure instead of the blocks of the previous sequential data structure.

[0068] Because the data described by the first data range and the second data range now most likely correspond to different disk addresses, the master node may also update the index of the first data range and the index of the second data range. The index of the first data range may be updated by sequentially reading the disk addresses of the appendages to the write sequential data structure. The index of the second data range may be updated based on the disk address returned from the virtual copy operation call according to step 310.

[0069] Subsequently, DFS processes subsequent file operations at the DFS node cluster with reference to the new sequential data structure. Based on the metadata stored at the master node, subsequent file writes appended to the new sequential data structure can be directed to disk operations of blocks stored at the managed storage of the block node.

[0070] Subsequently, the GC work process can reclaim the logical address corresponding to the previous sequential data structure as free space. According to garbage collection techniques known to those skilled in the art, such logical addresses can be reclaimed for new file operations.

[0071] Figure 4A and Figure 4B A flowchart of a virtual data replication method 400 according to an example embodiment of the present disclosure is shown.

[0072] At step 402, a distributed file system hosted at a distributed file system node cluster receives a virtual copy operation call having a data range, a source data structure, and a destination data structure as parameters.

[0073] Virtual copy operation calls can be referenced Figure 3 The call made in step 308. The data range may be any second data range in the second data range passed by the GC worker process; the source data structure may be a previous sequential data structure passed by the GC worker process (to be recycled after the virtual copy operation call), and the destination data structure may be a new sequential data structure.

[0074] According to an example embodiment of the present disclosure, the data range may identify a range of data to be virtually copied from at least a source data structure, a byte offset to start copying, and a byte length of the data to be copied.

[0075] According to an example embodiment of the present disclosure, the source data structure should include some number of sequential appends, such as log entries; each such sequential append should be readable, but each such sequential append should be closed to prevent further writing before being passed to the virtual copy operation call.

[0076] According to an example embodiment of the present disclosure, the destination data structure should include at least one sequential data structure such as a log that is open to write operations.

[0077] For the purpose of understanding the example embodiments of the present disclosure, it is assumed that the data range, source data structure, and destination data structure are each valid according to the above characteristics, and no invalid addresses, destinations not open for writing, etc. are involved.

[0078] At step 404, the DFS retrieves mapping metadata from a master node of the DFS node cluster, the mapping metadata including a mapping from a source data structure to blocks stored on a plurality of block nodes of the DFS node cluster.

[0079] The mapping metadata as described herein may relate to the location of each data block in a block node of a DFS node cluster. For example, the data block may be a block as described above. The mapping metadata may provide a mapping between: a file according to a file system hierarchy (as defined by DFS); and each data block constituting the file, the file including a copy of each data block as located in a block node.

[0080] According to an example embodiment of the present disclosure, one or more second data ranges passed to the virtual copy operation call are large enough in byte size so that the one or more second data ranges are expected to be mapped to blocks stored across more than one block node. Therefore, in order to avoid thread blocking workloads across a DFS node cluster, the virtual copy operation according to an example embodiment of the present disclosure can perform a virtual copy on the mapped blocks instead of performing a normal copy on the mapped blocks to write to the destination data structure.

[0081] Based on the mapping metadata, the parameters of the virtual copy operation call, such as data range, source data structure and destination data structure, can be checked for validity. In the event of any validity check failure, DFS can cause the virtual copy operation call to return failure.

[0082] At step 406, the DFS locates the chunk of the source data structure stored at the chunk node.

[0083] Each block of the source data structure can be located in this manner across any number of block nodes. Each block node can store one or more such located blocks.

[0084] At step 408, the DFS divides the data range into a plurality of logical address segments, each logical address segment corresponding to a positioning block stored on a block node.

[0085] Each logical address fragment includes a continuous address range within the data range corresponding to the positioning block stored on one block node. These block nodes can be the same or different. If multiple separate continuous address ranges are found in the data range, each continuous address range corresponds to a positioning block stored on the same block node, then each such continuous address area can be a different logical address fragment.

[0086] In step 410, the DFS forwards the replication command with the logical address segment parameter to the block node storing the corresponding positioning block.

[0087] The copy command may be, for example, a command in a protocol that supports inter-node messaging, implemented by the DFS API. For example, the copy command may be a command implemented by the DFS API (such as POSIX, FUSE, NFS, etc.) according to any implementation of a remote procedure call ("RPC") protocol. According to example embodiments of the present disclosure, support for the protocol should be implemented at each block node.

[0088] At step 412, the block node receives the copy command with the logical address segment parameter and converts the logical address segment to a source logical block address ("LBA") on the block node's hosted storage.

[0089] In this way, one or more LBAs (i.e., blocks according to the implementation of the block node) located at the managed storage of the block node are identified as source LBAs from which the virtual copy is to be performed. The block node can record the translation references to the source LBAs in the source list data structure. Each source LBA can include a reference to each positioning block as described above within each logical address segment, i.e., each block size aligned data block contains the source data to be copied.

[0090] As described above, LBAs may be mapped to disk addresses by a logical address mapper implemented on the managed storage of a block node. However, the conversion of logical address fragments to LBAs need not be performed with knowledge of the mapping to disk addresses.

[0091] At step 414, the block node reserves the destination LBA on the block node's managed storage corresponding to the source LBA.

[0092] Each destination LBA can be any LBA mapped to free disk space by a logical address mapper implemented on the managed storage of a block node. Just as a block node records references to source LBAs in a source list data structure, a block node can record references to destination LBAs in a destination list data structure.

[0093] At step 416, the block node passes a reference to the source LBA paired with a reference to the destination LBA to a remap call of the logical address mapper of the managed storage of the block node.

[0094] The remapping call may be made according to the remapping API call described above. The logical address mapper described above may implement the following mapping: mapping of LBAs such as the logical addresses of blocks in managed storage to disk addresses of underlying storage devices such as flash memory. Therefore, the logical address mapper may perform a remapping operation according to an example embodiment of the present disclosure by determining the disk address mapped to the source LBA and remapping the disk address to the destination LBA, thereby implementing a "virtual copy" from the perspective of a higher-level process (such as DFS) without actually relocating the data on the storage device underlying the managed storage. Subsequently, reference is made to Figure 5 Gives details of the logical address mapper that performs the remapping operation.

[0095] The logical address mapper may then be configured as follows: Figure 5 The method returns the destination LBA to the block node.

[0096] At step 418, the block node writes the destination LBA to the block node's remap block.

[0097] The destination LBA may be written to an existing chunk of the chunk node or a new chunk of the chunk node. In any case, as a result of the virtual copy operation, the remapped chunk replaces the previously located chunk as described above.

[0098] At step 420, the block node returns the remapped block to the DFS.

[0099] At step 422, the DFS merges the remapped chunks received from the block nodes to form a destination data structure and new mapping metadata.

[0100] The received blocks may be organized in logical address order to form logical address fragments, which may be reconstructed to form a destination data structure. Based on the merge, new mapping metadata may be generated, the new mapping metadata including a mapping from the destination data structure to blocks stored on a plurality of block nodes of the DFS node cluster (these block nodes are the block nodes that return the remapped blocks).

[0101] At step 424, the DFS returns new mapping metadata in response to the virtual copy operation call.

[0102] The new mapping metadata may provide the logical address corresponding to each second data range from the virtual copy operation call, as described above with reference to Figure 3 Therefore, Figure 3 The method can continue from here.

[0103] Figure 5A flowchart of a remapping method 500 performed at a logical address mapper of a managed storage of a block node according to an example embodiment of the present disclosure is shown.

[0104] At step 502, a logical address mapper of a managed storage of a block node receives a remap call having as a parameter a reference to a source logical block address ("LBA") paired with a reference to a destination LBA.

[0105] The remap call can be a reference Figure 4B The call is made in step 416 .

[0106] For the purpose of understanding the example embodiments of the present disclosure, it is assumed that for block addresses of managed storage, references to source LBAs and references to destination LBAs are valid and no invalid addresses are involved; and, in the absence of any LBA appearing more than once, references to source LBAs and references to destination LBAs are each unique within the parameters passed in the operation call.

[0107] The logical address mapper can identify multiple remapping calls by assigning sequence numbers to each remapping call in sequence.

[0108] The logical address mapper may store each source LBA-destination LBA pairing in memory.

[0109] At step 504, the logical address mapper maps the destination LBA to a disk address that is mapped to the source LBA that is paired with the destination LBA.

[0110] Based on the reference to the source LBA, the logical address mapper may determine the disk address to which the source LBA has been mapped by looking up the reference to the source LBA according to an index map stored in a memory of one or more storage devices as described above. According to an example embodiment of the present disclosure, the remapping operation as described herein seeks to map the disk address to a paired destination LBA instead of the source LBA, thereby replacing the source LBA.

[0111] At step 506, the logical address mapper writes a mapping of the destination LBA to the disk address of the managed storage.

[0112] Each mapping of a destination LBA to a disk address may be written to disk so that the consistency of the remapping operation is maintained across outages or power cycles. (Consistency in this context may ensure, for example, that a single entry will be valid and will reflect every previous entry, in accordance with data transaction guarantees known to those skilled in the art, such as atomicity, consistency, isolation, and durability ("ACID"). Each such mapping may be written as an entry to a local disk log implemented as a local file system.

[0113] At step 508, the logical address mapper submits the mapping of the destination LBA to the disk address to the file system of the managed storage.

[0114] For example, where each mapping is written as an entry in a local disk log, the logical address mapper may commit each recorded mapping by further recording a commit block following the recorded mapping. The commit block may contain any standardized data, depending on the disk log implementation of the local file system, which indicates that all entries before the commit block will not be restored after a system failure. (According to data transaction guarantees known to those skilled in the art, such as ACID, atomicity in this context may ensure that individual entries of the remap are not partially executed, so that the remap is not committed in a partial state in the event of a system failure.)

[0115] At step 510, the logical address mapper updates the index map of the managed storage with the mapping of the destination LBA to the disk address.

[0116] Since the results of the remapping operation have now been committed to managed storage, the index map can be safely updated to discard obsolete mappings from source LBAs to disk addresses. A TRIM operation implemented as known to those skilled in the art can then be applied to those blocks that were erased in response to storage space needs.

[0117] In the event of a system failure after step 508 but before step 510, depending on the implementation of the disk journal in the local file system, the journal that has been written to each map can be replayed to reconstruct the updates to be made to the index map.

[0118] The logical address mapper may then return the destination LBA to the block node, indicating the success of the remapping operation.

[0119] At step 512, the logical address mapper may write the index map to the checkpoint log.

[0120] The index map may be written at any time or periodically at any given interval to establish a reference to the most recent remapping call that has been completed. For example, as described above, the index map may be written to the checkpoint log in association with a sequence number. In the event that any mappings of destination LBAs to disk addresses written to managed storage have a lower sequence number than the most recent sequence number in the checkpoint log, these mappings may be discarded by garbage collection before the mappings are committed to managed storage.

[0121] Figure 6 Shows Figure 3 , Figure 4A , Figure 4B and Figure 5 A summary of the method, as in reference Figure 2 shown.

[0122] As above Figure 2 As shown in , a DFS node cluster 112 according to an example embodiment of the present disclosure includes a master node 116 and some number of block nodes 118. Each block node 118 stores some number of blocks 202, wherein any block stored on an individual block node 118 may be a copy of at least some number of other blocks stored on other individual block nodes 118. The master node 116 stores metadata 204 that provides a mapping between: a file according to a file system hierarchy (as defined by DFS); and each data block that constitutes the file, the file including a copy of each data block as located in the block node 118.

[0123] The client 206 of the DFS node cluster 112 may be any computing system that performs the following operations: performs computing tasks and utilizes the block nodes 118 of the DFS node cluster 112 as an extension of the local file system, utilizes the block nodes 118 of the DFS node cluster 112 in place of the local file system, utilizes the block nodes 118 of the DFS node cluster 112 with the ability to virtualize the local file system, etc. For example, the client 206 may be a server node 104 of the cloud network 102 in addition to the server nodes of the DFS node cluster 112, or may be a server node 104 of the DFS node cluster including the master node 116 and the block node 118.

[0124] Each block node 118 may be configured to perform disk operations such as reading data stored on the managed storage 120, writing data to the managed storage 120, copying data from one address on the managed storage 120 to another address on the managed storage 120 or copying data from one logical block on the managed storage 120 to another logical block on the managed storage 120, moving data from one address on the managed storage 120 to another address on the managed storage 120 or moving data from one logical block on the managed storage 120 to another logical block on the managed storage 120, etc. Each such disk operation may be performed by a call to the local file system 208 of the corresponding block node 118.

[0125] According to an example embodiment of the present disclosure, the DFS implemented at the DFS node cluster 112 can logically organize files as a log-structured file system, wherein each modification of data stored in the DFS can be stored as an addition to the header of a sequential data structure such as a log. Therefore, a call to the DFS to perform any type of logical file operation in which data is modified can cause the DFS to write an addition to the header of such a sequential data structure, which addition reflects the modification made by the file operation.

[0126] Implementing DFS as a log-structured file system may also cause subsequent appendages to replace previous appendages, rendering previous appendages obsolete, while reflecting more current versions of data written to the file system. Thus, DFS may also implement a garbage collection ("GC") coordinator process at the DFS node cluster 112 that controls GC worker processes. The GC coordinator process, along with the running GC worker processes, may traverse the sequential data structures of the DFS 212, identify data in obsolete appendages, and reclaim storage space occupied by obsolete appendages by writing subsequent appendages in the space. The GC worker process may typically be executed by a server node according to an example embodiment of the present disclosure as a background process that performs disk operations and has a lower priority than, for example, executing computer executable instructions that constitute a computing task.

[0127] As above Figure 3As shown in , based on the satisfaction of the GC trigger condition, the GC worker process running on the server node 104 of the DFS node cluster 112 is triggered to wake up by the GC coordinator process running on the coordinating server node. The GC worker process determines the first data range 604 and the second data range 606 of the sequential data structure 602 at the distributed file system 212. The GC worker process creates a new sequential data structure 608 and copies the data of each first data range 604 to the new sequential data structure 608. The GC worker process passes each second data range 606, the previous sequential data structure 602, and the new sequential data structure 608 to the virtual copy operation call of the DFS API as parameters.

[0128] Next, the distributed file system 212 hosted at the DFS node cluster 112 receives a virtual copy operation call having as parameters a data range 610, a source data structure 612, and a destination data structure 614. The DFS retrieves mapping metadata 204 from the master node 116 of the DFS node cluster 112, the mapping metadata 204 including a mapping from the source data structure 612 to blocks stored on a plurality of block nodes 118 of the DFS node cluster 112. The DFS locates blocks of the source data structure stored at the block nodes 118. The DFS divides the data range 610 into a plurality of logical address fragments 616, each logical address fragment 616 corresponding to a located block 618 stored on the block node 118. The DFS forwards the copy command with the logical address fragment parameter to the block node 118 storing the corresponding located block 618.

[0129] Next, the block node 118 receives the copy command with the logical address fragment 616 parameter and converts the logical address fragment 616 to a source logical block address (“source LBA 620”) on the managed storage 120 of the block node 118. The block node reserves a destination LBA 622 on the managed storage 120 of the block node 118 that corresponds to the source LBA 620. The block node 118 passes a reference to the source LBA 620 paired with a reference to the destination LBA 622 to a remap call of a logical address mapper 624 of the managed storage 120 of the block node 118.

[0130] Next, the logical address mapper 624 of the managed storage 120 of the block node 118 receives the remap call having as a parameter a reference to the source LBA 620 paired with the reference to the destination LBA 622. The logical address mapper maps the destination LBA 622 to a disk address 628 that is mapped to the source LBA 620 paired with the destination LBA 622. The logical address mapper writes the mapping of the destination LBA 622 to the disk address 628 of the managed storage 120. The logical address mapper commits the mapping of the destination LBA 622 to the disk address 628 to the local file system 208 of the managed storage 120. The logical address mapper updates the index map 630 of the managed storage 120 with the mapping of the destination LBA 622 to the disk address 628. The logical address mapper writes the index map 630 to the checkpoint log.

[0131] Next, the block node 118 writes the destination LBA 622 to the remap chunk 626 of the block node 118. The block node 118 returns the remap chunk 626 to the DFS.

[0132] Next, the DFS merges the remapped chunks 626 received from the block nodes 118 to form the destination data structure 614 and new mapping metadata 632. The DFS returns the new mapping metadata 632 in response to the virtual copy operation call.

[0133] Next, the GC worker process obtains the logical address corresponding to each second data range 606 based on the virtual copy operation call. The GC coordinator process notifies the master node 116 of the DFS node cluster 112 of the new sequential data structure 608.

[0134] Fig. 7A and Figure 7B An example distributed file system 700 is shown for implementing the above-described processes and methods of virtual data replication.

[0135] The techniques and mechanisms described herein can be implemented by multiple instances of system 700 and any other computing devices, systems and / or environments. System 700 can be a single computing system or edge host providing physical computing resources or virtual computing resources known to those skilled in the art. Examples thereof include at least one node of a DFS node cluster as described herein. Fig. 7A and Figure 7BThe system 700 shown in is only an example of a system and is not intended to impose any limitation on the scope of use or functionality of any computing device used to perform the processes and / or procedures described above. Other known computing devices, systems, environments, and / or configurations that may be suitable for use with the embodiments include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, implementations using field programmable gate arrays ("FPGAs") and application specific integrated circuits ("ASICs"), and the like.

[0136] The system 700 may include one or more processors 702 and a system memory 704 communicatively coupled to the processor(s) 702. The processor(s) 702 and the system memory 704 may be physical, or may be virtualized and / or distributed. The processor(s) 702 may execute one or more modules and / or processes to enable the processor(s) 702 to perform various functions. In an embodiment, the processor(s) 702 may include a central processing unit (“CPU”), a graphics processing unit (“GPU”), both a CPU and a GPU, or other processing units or components known in the art. Alternatively, each of the processor(s) 702 may have its own local memory, which may also store program modules, program data, and / or one or more operating systems.

[0137] System memory 704 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, micro hard drive, memory card, etc.), or some combination thereof, depending on the exact configuration and type of system 700. System memory 704 may include one or more computer executable modules 706 that may be executed by processor(s) 702.

[0138] The module 706 may include, but is not limited to, a call receiving module 708, a metadata retrieval module 710, a block locating module 712, a range dividing module 714, a command forwarding module 716, a block merging module 718, a metadata returning module 720, and a garbage collection module 722. The garbage collection module 722 may be stored on the same system 700 as the rest of the above modules, or may be stored on a different instance of the system 700. The garbage collection module 722 may also include a wake-up submodule 724, a range determining submodule 726, a data structure creating submodule 728, an operation call delivering submodule 730, an address receiving submodule 732, and a master notification submodule 734.

[0139] The call receiving module 708 may be configured as described above. Figure 4A and Figure 4B As described, a virtual copy operation call is received.

[0140] The metadata retrieval module 710 may be configured as described above. Figure 4A and Figure 4B As described, the mapping metadata is retrieved from the master node of the DFS node cluster.

[0141] The block location module 712 may be configured as described above with reference to Figure 4A and Figure 4B As described, a block of a source data structure stored at a block node is located.

[0142] The range partitioning module 714 may be configured as described above with reference to Figure 4A and Figure 4B As described, the data range is divided into a plurality of logical address segments.

[0143] The command forwarding module 716 may be configured as described above with reference to Figure 4A and Figure 4B As described, the replication command is forwarded to the block nodes.

[0144] The block merging module 718 may be configured as described above with reference to Figure 4A and Figure 4B As described, merge remapped blocks received from block nodes.

[0145] The metadata return module 720 may be configured as described above. Figure 4A and Figure 4B As described, in response to a virtual copy operation call, new mapping metadata is returned.

[0146] The wake-up submodule 724 may be configured as described above. Figure 3 As described, wake up the GC worker thread.

[0147] The range determination submodule 726 may be configured as described above with reference to Figure 3 As described, a first data range and a second data range of a sequential data structure are determined at a local file system.

[0148] The data structure creation submodule 728 may be configured as described above with reference to Figure 3 As described, a new sequential data structure is created.

[0149] The operation call transfer submodule 730 may be configured as described above. Figure 3 As described, pass the virtual copy operation call.

[0150] The address receiving submodule 732 may be configured as described above with reference to Figure 3As described, a logical address is received.

[0151] The main notification submodule 734 can be configured as described above. Figure 3 As described, notify the master node.

[0152] The system 700 may also include an input / output (I / O) interface 740 and a communication interface 750, which enables the system 700 to communicate with the system via a network (such as the one referenced above). Figure 1 The network may include the Internet, wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency ("RF"), infrared and other wireless media.

[0153] Some or all of the operations of the method described above can be performed by executing computer-readable instructions as defined below stored on a computer-readable storage medium. The term "computer-readable instructions" as used in the specification and claims includes routines, applications, application modules, program modules, programs, components, data structures, algorithms, etc. Computer-readable instructions can be implemented on a variety of system configurations, including single-processor systems or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based systems, programmable consumer electronics, combinations thereof, etc.

[0154] Computer-readable storage media may include volatile memory (such as random access memory ("RAM")) and / or non-volatile memory (such as read-only memory ("ROM"), flash memory, etc.). Computer-readable storage media may also include additional removable storage and / or non-removable storage, including but not limited to flash memory, magnetic storage, optical storage, and / or tape storage, which may provide non-volatile storage of computer-readable instructions, data structures, program modules, etc.

[0155] Non-transitory computer-readable storage media are examples of computer-readable media. Computer-readable media include at least two types of computer-readable media, namely computer-readable storage media and communication media. Computer-readable storage media include volatile and non-volatile media, removable and non-removable media, which are implemented in any process or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, phase change memory ("PRAM"), static random access memory ("SRAM"), dynamic random access memory ("DRAM"), other types of random access memory ("RAM"), read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), flash memory or other memory technology, compact disc read-only memory ("CD-ROM"), digital versatile disk ("DVD") or other optical storage, cassette, tape, disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media can embody computer-readable instructions, data structures, program modules or other data in modulated data signals (such as carrier waves) or other transmission mechanisms. As defined herein, computer readable storage media do not include communications media.

[0156] Computer readable instructions stored on one or more non-transitory computer readable storage media, when executed by one or more processors, may perform the above-referenced Figures 1 to 6 The operations described. Generally, computer-readable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement a process.

[0157] Fig. 8A and Figure 8B An example server node system 800 is shown for implementing the above-described processes and methods of virtual data replication.

[0158] The techniques and mechanisms described herein can be implemented by multiple instances of system 800 and any other computing devices, systems and / or environments. System 800 can be a single computing system or edge host providing physical computing resources or virtual computing resources known to those skilled in the art. Examples thereof include at least one node of a DFS node cluster as described herein. Fig. 8A and Figure 8BThe system 800 shown in is only an example of a system and is not intended to impose any limitation on the scope of use or functionality of any computing device used to perform the processes and / or procedures described above. Other known computing devices, systems, environments, and / or configurations that may be suitable for use with the embodiments include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, implementations using field programmable gate arrays ("FPGAs") and application specific integrated circuits ("ASICs"), and the like.

[0159] The system 800 may include one or more processors 802 and a system memory 804 communicatively coupled to the processor(s) 802. The processor(s) 802 and the system memory 804 may be physical, or may be virtualized and / or distributed. The processor(s) 802 may execute one or more modules and / or processes to enable the processor(s) 802 to perform various functions. In an embodiment, the processor(s) 802 may include a central processing unit ("CPU"), a graphics processing unit ("GPU"), both a CPU and a GPU, or other processing units or components known in the art. Alternatively, each of the processor(s) 802 may have its own local memory, which may also store program modules and program data.

[0160] System memory 804 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, micro hard drive, memory card, etc.), or some combination thereof, depending on the exact configuration and type of system 800. System memory 804 may include one or more computer executable modules 806 that may be executed by processor(s) 802.

[0161] The module 806 may include, but is not limited to, a block node module 808 and a logical address mapper module 810. The block node module 808 may include a command receiving submodule 812, a reservation submodule 814, a remap call delivery submodule 816, a block writing submodule 818, and a block returning submodule 820. The logical address mapper module 810 may include a call receiving submodule 822, a mapping submodule 824, a mapping writing submodule 826, a mapping committing submodule 828, an index map updating submodule 830, and an index map writing submodule 832.

[0162] The command receiving submodule 812 may be configured as described above. Figure 4A and Figure 4B As described, a copy command is received.

[0163] The retention submodule 814 may be configured as described above with reference to Figure 4A and Figure 4B As described, the destination LBA is preserved on the managed storage.

[0164] The remap call transfer submodule 816 may be configured as described above with reference to Figure 4A and Figure 4B As described, a reference to the source LBA paired with a reference to the destination LBA is passed to the remap call.

[0165] The block writing submodule 818 may be configured as described above with reference to Figure 4A and Figure 4B As described, the destination LBA is written to the remap block.

[0166] The block return submodule 820 may be configured as described above. Figure 4A and Figure 4B As described, the remapped blocks are returned to the DFS.

[0167] The call receiving submodule 822 may be configured as described above. Figure 5 As described, a remap call is received having a reference to a source LBA paired with a reference to a destination LBA.

[0168] The mapping submodule 824 may be configured as described above with reference to Figure 5 As described, a destination LBA is mapped to a disk address that is mapped to a source LBA that is paired with the destination LBA.

[0169] The mapping write submodule 826 can be configured as described above. Figure 5 As described, a mapping of destination LBAs to disk addresses is written to managed storage.

[0170] The mapping submission submodule 828 may be configured as described above with reference to Figure 5 As described, the mapping of destination LBAs to disk addresses is submitted to the local file system of the hosted storage.

[0171] The index map update submodule 830 may be configured as described above. Figure 5 As described, the index map of the managed storage is updated using the mapping of the destination LBA to the disk address.

[0172] The index map writing submodule 832 may be configured as described above. Figure 5 Write the index map to the checkpoint log as described.

[0173] The system 800 may also include an input / output (I / O) interface 840 and a communication interface 850, which enables the system 800 to communicate with the system via a network (such as the one referenced above). Figure 1 The network may include the Internet, wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency ("RF"), infrared and other wireless media.

[0174] Some or all of the operations of the method described above can be performed by executing computer-readable instructions as defined below stored on a computer-readable storage medium. The term "computer-readable instructions" as used in the specification and claims includes routines, applications, application modules, program modules, programs, components, data structures, algorithms, etc. Computer-readable instructions can be implemented on a variety of system configurations, including single-processor systems or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based systems, programmable consumer electronics, combinations thereof, etc.

[0175] Computer-readable storage media may include volatile memory (such as random access memory ("RAM")) and / or non-volatile memory (such as read-only memory ("ROM"), flash memory, etc.). Computer-readable storage media may also include additional removable storage and / or non-removable storage, including but not limited to flash memory, magnetic storage, optical storage, and / or tape storage, which may provide non-volatile storage of computer-readable instructions, data structures, program modules, etc.

[0176] Non-transitory computer-readable storage media are examples of computer-readable media. Computer-readable media include at least two types of computer-readable media, namely computer-readable storage media and communication media. Computer-readable storage media include volatile and non-volatile media, removable and non-removable media, which are implemented in any process or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, phase change memory ("PRAM"), static random access memory ("SRAM"), dynamic random access memory ("DRAM"), other types of random access memory ("RAM"), read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), flash memory or other memory technology, compact disc read-only memory ("CD-ROM"), digital versatile disk ("DVD") or other optical storage, cassette, tape, disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media can embody computer-readable instructions, data structures, program modules or other data in modulated data signals (such as carrier waves) or other transmission mechanisms. As defined herein, computer readable storage media do not include communications media.

[0177] Computer readable instructions stored on one or more non-transitory computer readable storage media, when executed by one or more processors, may perform the above-referenced Figures 1 to 6 The operations described. Generally, computer-readable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement a process.

[0178] 9A to 9D An example multi-tier system 900 is shown for implementing the above-described processes and methods of virtual data replication.

[0179] The techniques and mechanisms described herein and implemented on the distributed file system 700 and the server node system 800 can each be implemented on an instance of the multi-layer system 900, resulting in the implementation of each "layer" of functionality conceptualized by the distributed file system 700 and the server node 800 on the same system. The system 900 can be a single computing system or edge host providing physical computing resources or virtual computing resources known to those skilled in the art. Examples thereof include at least one node of a DFS node cluster as described herein. Figures 9A to 9DThe system 900 shown in is only an example of a system and is not intended to impose any limitation on the scope of use or functionality of any computing device used to perform the processes and / or procedures described above. Other known computing devices, systems, environments, and / or configurations that may be suitable for use with the embodiments include, but are not limited to, personal computers, server computers, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, implementations using field programmable gate arrays ("FPGAs") and application specific integrated circuits ("ASICs"), and the like.

[0180] The system 900 may include one or more processors 902 and a system memory 904 communicatively coupled to the processor(s) 902. The processor(s) 902 and the system memory 904 may be physical, or may be virtualized and / or distributed. The processor(s) 902 may execute one or more modules and / or processes to enable the processor(s) 902 to perform various functions. In an embodiment, the processor(s) 902 may include a central processing unit ("CPU"), a graphics processing unit ("GPU"), both a CPU and a GPU, or other processing units or components known in the art. Alternatively, each of the processor(s) 902 may have its own local memory, which may also store program modules and program data.

[0181] System memory 904 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, micro hard drive, memory card, etc.), or some combination thereof, depending on the exact configuration and type of system 900. System memory 904 may include one or more computer executable modules 906 that may be executed by processor(s) 902.

[0182] The module 906 may include, but is not limited to, a call receiving module 908, a metadata retrieval module 910, a block positioning module 912, a range partitioning module 914, a command forwarding module 916, a block merging module 918, a metadata returning module 920, and a garbage collection module 922. The garbage collection module 922 may be stored on the same system 900 as the rest of the above modules, or may be stored on a different instance of the system 900. The garbage collection module 922 may also include a wake-up submodule 924, a range determination submodule 926, a data structure creation submodule 928, an operation call delivery submodule 930, an address reception submodule 932, and a master notification submodule 934.

[0183] The module 906 may also include, but is not limited to, a block node module 936 and a logical address mapper module 938. The block node module 936 may include a command receiving submodule 940, a reservation submodule 942, a remap call delivery submodule 944, a block writing submodule 946, and a block returning submodule 948. The logical address mapper module 938 may include a call receiving submodule 950, a mapping submodule 952, a mapping writing submodule 954, a mapping submission submodule 956, an index map updating submodule 958, and an index map writing submodule 960.

[0184] The call receiving module 908 may be configured as described above. Figure 4A and Figure 4B As described, a virtual copy operation call is received.

[0185] The metadata retrieval module 910 may be configured as described above. Figure 4A and Figure 4B As described, the mapping metadata is retrieved from the master node of the DFS node cluster.

[0186] The block location module 912 may be configured as described above. Figure 4A and Figure 4B As described, a block of a source data structure stored at a block node is located.

[0187] The range partitioning module 914 can be configured as described above. Figure 4A and Figure 4B As described, the data range is divided into a plurality of logical address segments.

[0188] The command forwarding module 916 may be configured as described above. Figure 4A and Figure 4B As described, the replication command is forwarded to the block nodes.

[0189] The block merging module 918 may be configured as described above with reference to Figure 4A and Figure 4B As described, merge remapped blocks received from block nodes.

[0190] The metadata return module 920 may be configured as described above. Figure 4A and Figure 4B As described, in response to a virtual copy operation call, new mapping metadata is returned.

[0191] The wake-up submodule 924 may be configured as described above. Figure 3 As described, wake up the GC worker thread.

[0192] The range determination submodule 926 may be configured as described above with reference to Figure 3As described, a first data range and a second data range of a sequential data structure are determined at a local file system.

[0193] The data structure creation submodule 928 may be configured as described above with reference to Figure 3 As described, a new sequential data structure is created.

[0194] The operation call transfer submodule 930 may be configured as described above. Figure 3 As described, pass the virtual copy operation call.

[0195] The address receiving submodule 932 may be configured as described above with reference to Figure 3 As described, a logical address is received.

[0196] The main notification submodule 934 can be configured as described above. Figure 3 As described, notify the master node.

[0197] The command receiving submodule 940 may be configured as described above. Figure 4A and Figure 4B As described, a copy command is received.

[0198] The retention submodule 942 may be configured as described above with reference to Figure 4A and Figure 4B As described, the destination LBA is preserved on the managed storage.

[0199] The remap call transfer submodule 944 may be configured as described above with reference to Figure 4A and Figure 4B As described, a reference to the source LBA paired with a reference to the destination LBA is passed to the remap call.

[0200] The block write submodule 946 may be configured as described above with reference to Figure 4A and Figure 4B As described, the destination LBA is written to the remap block.

[0201] The block return submodule 948 can be configured as described above. Figure 4A and Figure 4B As described, the remapped blocks are returned to the DFS.

[0202] The call receiving submodule 950 may be configured as described above. Figure 5 As described, a remap call is received having a reference to a source LBA paired with a reference to a destination LBA.

[0203] The mapping submodule 952 may be configured as described above with reference to Figure 5As described, a destination LBA is mapped to a disk address that is mapped to a source LBA that is paired with the destination LBA.

[0204] The mapping write submodule 954 can be configured as described above. Figure 5 As described, a mapping of destination LBAs to disk addresses is written to managed storage.

[0205] The mapping submission submodule 956 may be configured as described above with reference to Figure 5 As described, the mapping of destination LBAs to disk addresses is submitted to the local file system of the hosted storage.

[0206] The index map update submodule 958 may be configured as described above with reference to Figure 5 As described, the index map of the managed storage is updated using the mapping of the destination LBA to the disk address.

[0207] The index map writing submodule 960 may be configured as described above with reference to Figure 5 Write the index map to the checkpoint log as described.

[0208] The system 900 may also include an input / output (I / O) interface 970 and a communication interface 980, which enables the system 900 to communicate with the system via a network (such as the one referenced above). Figure 1 The network may include the Internet, wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency ("RF"), infrared and other wireless media.

[0209] Some or all of the operations of the method described above can be performed by executing computer-readable instructions as defined below stored on a computer-readable storage medium. The term "computer-readable instructions" as used in the specification and claims includes routines, applications, application modules, program modules, programs, components, data structures, algorithms, etc. Computer-readable instructions can be implemented on a variety of system configurations, including single-processor systems or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based systems, programmable consumer electronics, combinations thereof, etc.

[0210] Computer-readable storage media may include volatile memory (such as random access memory ("RAM")) and / or non-volatile memory (such as read-only memory ("ROM"), flash memory, etc.). Computer-readable storage media may also include additional removable storage and / or non-removable storage, including but not limited to flash memory, magnetic storage, optical storage, and / or tape storage, which may provide non-volatile storage of computer-readable instructions, data structures, program modules, etc.

[0211] Non-transitory computer-readable storage media are examples of computer-readable media. Computer-readable media include at least two types of computer-readable media, namely computer-readable storage media and communication media. Computer-readable storage media include volatile and non-volatile media, removable and non-removable media, which are implemented in any process or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, phase change memory ("PRAM"), static random access memory ("SRAM"), dynamic random access memory ("DRAM"), other types of random access memory ("RAM"), read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), flash memory or other memory technology, compact disc read-only memory ("CD-ROM"), digital versatile disk ("DVD") or other optical storage, cassette, tape, disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media can embody computer-readable instructions, data structures, program modules or other data in modulated data signals (such as carrier waves) or other transmission mechanisms. As defined herein, computer readable storage media do not include communications media.

[0212] It should be further understood that hosted storage as described herein may be hosted at least in part at multi-tier system 900 .

[0213] Computer readable instructions stored on one or more non-transitory computer readable storage media, when executed by one or more processors, may perform the above-referenced Figures 1 to 6 The operations described. Generally, computer-readable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement a process.

[0214] Through the above technical solution, the present disclosure provides a virtual copy operation, which improves the support for the preemptive garbage collection strategy at the distributed file system. A garbage collection process is provided, which preemptively performs copying of data in the log structure file system to avoid heavy thread blocking workload. In order to prevent the thread blocking write operation from being distributed across the nodes of the distributed file system node cluster, and thus avoid the decline of computing performance, a virtual copy operation is provided in the DFS API, which locates each block node storing the block to be virtually copied based on the master node metadata, and calls the remapping API of the logical address mapper, which is provided by the local file system of the block node. The logical address mapper of each block node performs remapping of the disk address from the source logical block address to the destination logical block address without relocating the data from one disk address to another disk address. The results of these remappings can be stored as metadata at the master node, replacing the previously mapped metadata. In this way, the copy operation necessary to form the basis of the preemptive garbage collection strategy is virtually executed, bypassing a large amount of disk-level workload across the DFS node cluster.

[0215] Example Item

[0216] A. A method comprising: determining, at a distributed file system ("DFS") node cluster, a first data range and a second data range of a sequential data structure; creating a new sequential data structure and copying data of each first data range to the new sequential data structure; and passing each second data range, the sequential data structure, and the new sequential data structure to a virtual copy operation call of the DFS as parameters.

[0217] B. The method according to paragraph A also includes: receiving a logical address corresponding to each second data range from a virtual copy operation call.

[0218] C. The method described in paragraph B also includes: notifying the master node of the DFS node cluster of the new sequential data structure.

[0219] D. A method, comprising: receiving a virtual copy operation call at a DFS node cluster, the virtual copy operation call having a data range, a source data structure, and a destination data structure as parameters; retrieving mapping metadata from a master node of the DFS node cluster, the mapping metadata comprising a mapping from the source data structure to blocks stored on multiple block nodes of the DFS node cluster; locating blocks of the source data structure stored at the block node; dividing the data range into multiple logical address fragments, each logical address fragment corresponding to a located block stored on the block node; and forwarding a copy command with the logical address fragment parameters to the block node storing the corresponding located block.

[0220] E. The method according to paragraph D also includes: merging the remapped blocks received from the block nodes to form a destination data structure and new mapping metadata; and returning the new mapping metadata in response to a virtual copy operation call.

[0221] F. A method comprising: receiving a copy command with a logical address fragment parameter from a distributed file system ("DFS") and converting the logical address fragment to a source logical block address ("source LBA") on managed storage; retaining a destination LBA on managed storage corresponding to the source LBA; and passing a reference to the source LBA paired with a reference to the destination LBA to a remap call of a logical address mapper of the managed storage.

[0222] G. The method according to paragraph F also includes: receiving a remapping call by a logical address mapper, the remapping call having a reference to a source LBA paired with a reference to a destination LBA as a parameter; and updating an index map of the managed storage by the logical address mapper using a mapping of the destination LBA to a disk address.

[0223] H. The method according to paragraph G, wherein the destination LBA is mapped to the disk address by the following operation: the destination LBA is mapped to a disk address by a logical address mapper, and the disk address is mapped to a source LBA paired with the destination LBA.

[0224] I. The method described in paragraph H also includes: the logical address mapper writes the mapping of the destination LBA to the disk address of the managed storage; and the logical address mapper submits the mapping of the destination LBA to the disk address to the local file system of the managed storage.

[0225] J. The method according to paragraph G also includes: the logical address mapper assigning a serial number to the remapping call; and the logical address mapper writing the index map to the checkpoint log in association with the serial number.

[0226] K. The method according to paragraph F also includes: writing the destination LBA to the remap block and returning the remap block to the DFS.

[0227] L. A method according to paragraph F, wherein the managed storage includes at least a flash storage device.

[0228] M. A method, comprising: one or more processors; and a memory communicatively connected to the one or more processors; the memory storing a computer executable module capable of being executed by the one or more processors, the computer executable module performing associated operations when executed by the one or more processors, the computer executable module comprising a block node module, the block node module further comprising: a command receiving submodule, a reservation submodule, and a remapping call passing submodule; the command receiving submodule being configured to receive a copy command having a logical address fragment parameter from a distributed file system ("DFS"), and convert the logical address fragment into a source logical block address ("source LBA") on a managed storage; the reservation submodule being configured to reserve a destination LBA on the managed storage corresponding to the source LBA; the remapping call passing submodule being configured to pass a reference to the source LBA paired with a reference to the destination LBA to a remapping call of a logical address mapper of the managed storage.

[0229] N. The system according to paragraph M further includes a logical address mapper module, which includes: a call receiving submodule and an index map updating submodule; the call receiving submodule is configured to receive a remapping call, which has a reference to a source LBA paired with a reference to a destination LBA as a parameter; the index map updating submodule is configured to update the index map of the managed storage using the mapping of the destination LBA to the disk address.

[0230] O. A system according to paragraph N, wherein the logical address mapper module further includes a mapping submodule configured to map a destination LBA to a disk address that is mapped to a source LBA paired with the destination LBA.

[0231] P. A system according to paragraph O, wherein the logical address mapper module further comprises: a mapping write submodule and a mapping commit submodule; the mapping write submodule is configured to write a mapping of the destination LBA to the disk address of the managed storage; the mapping commit submodule is configured to commit the mapping of the destination LBA to the disk address to the local file system of the managed storage.

[0232] Q. A system according to paragraph N, wherein the logical address mapper module is further configured to assign sequence numbers to remapping calls and write the index map to the checkpoint log in association with the sequence numbers.

[0233] R. A system according to paragraph M, wherein the block node module further comprises: a block write submodule and a block return submodule; the block write submodule is configured to write the destination LBA to the remapped block; and the block return submodule is configured to return the remapped block to the DFS.

[0234] S. A system according to paragraph M, wherein the managed storage includes at least a flash storage device.

[0235] T. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, which, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving a copy command with a logical address fragment parameter from a distributed file system ("DFS") and converting the logical address fragment to a source logical block address ("source LBA") on a managed storage; reserving a destination LBA on the managed storage corresponding to the source LBA; and passing a reference to the source LBA paired with a reference to the destination LBA to a remapping call of a logical address mapper of the managed storage.

[0236] U. A computer-readable storage medium according to paragraph T, wherein the operation also includes: receiving a remapping call by a logical address mapper, the remapping call having a reference to a source LBA paired with a reference to a destination LBA as a parameter; and updating an index map of the managed storage by the logical address mapper using a mapping of the destination LBA to a disk address.

[0237] V. A computer-readable storage medium according to paragraph U, wherein the destination LBA is mapped to the disk address in the following manner: the destination LBA is mapped to a disk address by a logical address mapper, and the disk address is mapped to a source LBA paired with the destination LBA.

[0238] W. A computer-readable storage medium according to paragraph V, wherein the operation further comprises: writing, by a logical address mapper, a mapping of a destination LBA to a disk address of managed storage; and submitting, by the logical address mapper, the mapping of a destination LBA to a disk address to a local file system of managed storage.

[0239] X. A computer-readable storage medium according to paragraph U, wherein the operation also includes: assigning a serial number to the remapping call by the logical address mapper; and writing the index map to the checkpoint log in association with the serial number by the logical address mapper.

[0240] Y. A computer-readable storage medium according to paragraph T, wherein the operation further includes: writing the destination LBA to the remap block and returning the remap block to the DFS.

[0241] Z. A computer-readable storage medium according to paragraph T, wherein the managed storage includes at least a flash memory storage device.

[0242] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

1. A virtual copy method, comprising: During the virtual copy operation performed by the distributed file system DFS, receiving a copy command from the DFS; Passing a remap call to a logical address mapper of the managed storage based on a conversion parameter of the copy command; The logical address mapper updates a mapping to a disk address at an index map of the managed storage; as well as Returning the remapped block to the DFS, so that the DFS completes the virtual copy operation; Updating, by the logical address mapper, the mapping to the disk address at the index map of the hosted storage includes remapping the disk address from a reference to an anchor block of source LBA parameters to a reference to a reserved block of destination LBA parameters.

2. The method according to claim 1, wherein: The virtual copy operation is invoked through a disk operation of a storage service hosted on a server node of a DFS node cluster implementing the DFS.

3. The method according to claim 2, wherein: The virtual copy operation invoked by a disk operation includes a GC worker process that determines a data range greater than a virtual copy threshold after the disk operation.

4. The method according to claim 3, wherein: The data range parameter of the copy command includes a plurality of logical address segments, each of the plurality of logical address segments corresponding to a location block at the hosted storage.

5. The method according to claim 4, wherein: A source LBA parameter of the remap call includes a list recording each located block in the plurality of logical address segments, and a destination LBA parameter of the remap call includes a plurality of reserved blocks at the managed storage.

6. The method according to claim 1, further comprising: The logical address mapper submits the updated mapping to the disk address to the local file system of the managed storage; assigning, by the logical address mapper, a sequence number to the remapping call; as well as The index map is written into a checkpoint log by the logical address mapper in association with the sequence number.

7. A virtual replication system, comprising: one or more processors; storage hosted at least in part at said system; as well as A memory communicatively coupled to the one or more processors, the memory storing a computer executable module executable by the one or more processors, wherein when the computer executable module is executed by the one or more processors, the computer executable module performs associated operations, the computer executable module comprising: a storage service hosted at the system, the storage service configured to perform a disk operation invoked by the virtual copy operation; A call receiving module, wherein the call receiving module is configured to receive the virtual copy operation call; A command forwarding module, wherein the command forwarding module is configured to forward the copy command; A block node module, wherein the block node module further comprises: A command receiving submodule, wherein the command receiving submodule is configured to receive a copy command from the command forwarding module; a remapping call delivery submodule configured to deliver a remapping call to a logical address mapper module of the system based on a conversion parameter of the copy command; and a block returning submodule, the block returning submodule being configured to return the remapped block; and A logical address mapper module, the logical address mapper module further comprising: An index map update submodule, wherein the index map update submodule is configured to update the mapping to the disk address at the index map of the managed storage; the index map update submodule is configured to update the mapping to the disk address by remapping the disk address from a reference to a positioning block of a source LBA parameter to a reference to a reserved block of a destination LBA parameter.

8. The system according to claim 7, further comprising a garbage collection module, wherein the garbage collection module further comprises: A range determination submodule is configured to determine a GC working process, which determines a data range greater than a virtual copy threshold after the disk operation.

9. The system according to claim 8, wherein: The data range parameter of the copy command includes a plurality of logical address segments, each of the plurality of logical address segments corresponding to a location block at the hosted storage.

10. The system according to claim 9, wherein: The command receiving submodule is further configured to record a list, wherein the list records each positioning block in the plurality of logical address fragments, and the source LBA parameter of the remapping call includes the list recording each positioning block.

11. The system according to claim 10, wherein: The logical address mapper module further includes a reservation submodule configured to reserve a plurality of reserved blocks at the managed storage, and a destination LBA parameter of the remap call includes the plurality of reserved blocks.

12. The system according to claim 7, wherein: The logical address mapper module further includes a mapping submission submodule configured to submit the updated mapping to the disk address to the local file system of the managed storage; and The logical address mapper module is further configured to assign a sequence number to the remapping call and write the index map to a checkpoint log in association with the sequence number.

13. A computer-readable storage medium storing computer-readable instructions executable by one or more processors, wherein when the computer-readable instructions are executed by the one or more processors, the computer-readable instructions cause the one or more processors to perform operations, the operations comprising: During the virtual copy operation performed by the distributed file system DFS, receiving a copy command from the DFS; Passing a remapping call to a logical address mapper of the managed storage of the DFS based on a conversion parameter of the copy command; The logical address mapper updates a mapping to a disk address at an index map of the managed storage; as well as Returning the remapped block to the DFS, so that the DFS completes the virtual copy operation; Updating, by the logical address mapper, the mapping to the disk address at the index map of the hosted storage includes remapping the disk address from a reference to an anchor block of source LBA parameters to a reference to a reserved block of destination LBA parameters.

14. The computer-readable storage medium of claim 13, wherein: The virtual copy operation is invoked through a disk operation of a storage service hosted on a server node of a DFS node cluster implementing the DFS.

15. The computer-readable storage medium of claim 14, wherein: The virtual copy operation invoked by a disk operation includes a GC worker process that determines a data range greater than a virtual copy threshold after the disk operation.

16. The computer-readable storage medium of claim 15, wherein: The data range parameter of the copy command includes a plurality of logical address segments, each of the plurality of logical address segments corresponding to a location block at the hosted storage.

17. The computer-readable storage medium of claim 16, wherein: A source LBA parameter of the remap call includes a list recording each located block in the plurality of logical address segments, and a destination LBA parameter of the remap call includes a plurality of reserved blocks at the managed storage.