Garbage Collection in Data Storage Systems

Abstract

Garbage collection is performed for a virtualized storage system whose virtual address space is addressed by extents. Valid data in a source extent is copied to a destination extent via a cache. Once all valid data in a source extent has been copied to one or more destination extents, the source extent is reclaimed. A source extent is released for reuse only after one or more destination extents that received valid data copied from the source extent are determined to be full, and valid data copied from the source extent to the destination extent via a cache is flushed from the cache.

Description

[Technical field]

[0001] The present invention relates to garbage collection in data storage systems. [Background technology]

[0002] A common strategy for performing writes in data storage systems is to always write to a new range of data in the data domain. This strategy has the advantage that it is possible to update a Redundant Array of Independent Disks (RAID) array without requiring a read from the disk.

[0003] A typical "write" would be performed as follows: An input-output (IO) is received for a given volume, with an address, size, and some data. The storage allocator assigns that volume to a storage location on the physical domain. The IO path then writes to that location on the physical domain. Once the write is complete, the forward and reverse lookup structures are updated and the IO is completed. As part of the forward lookup update, the journal is also updated.

[0004] A typical "read" would look at the forward lookup structure and identify where on the data domain the data is located. The data domain is then read and the data is returned to the user. Summary of the Invention

[0005] Various embodiments are directed to a method of garbage collection in a storage system in which a virtual address space is mapped to a physical address space, where the virtual address space is addressed in extents, and the storage system has one or more processors, a plurality of storage devices, and a cache memory. The method includes selecting a first destination extent as a current destination extent for receiving garbage collected data, and selecting the first source extent as a current source extent to be garbage collected, where the current source extent includes one or more portions of valid data. The method also includes determining whether the destination extent selected as the current destination extent for receiving the garbage collected data is full. The method includes determining whether the source extent selected as the current source extent to be garbage collected includes one or more portions of valid data that have not been copied to the destination extent. Additionally, the method includes, in response to determining that the current destination extent for receiving the garbage collected data is not full and determining that the current source extent to be garbage collected includes one or more portions of valid data that have not been copied to the destination extent, copying one of the portions of valid data that have not been copied to the destination extent from the current source extent to the current destination extent via the cache memory. Additionally, the method includes, in response to determining that the current destination extent for receiving the garbage collected data is full, flushing the valid data to be copied to the current destination extent from the cache memory via the cache memory.Additionally, following the step of flushing the valid data that has been moved from the cache memory to the current destination extent, the method includes the step of releasing all valid data that has been copied to the current destination extent via the cache memory so that it is available for writing to a physical address space that is mapped to the virtual address space of the one or more source extents.

[0006] Various embodiments of the garbage collection method may include, in response to determining that the current destination extent for receiving the garbage collected data is full, selecting a second destination extent as the current destination extent for receiving the garbage collected data.

[0007] Various embodiments of the garbage collection method further include, in response to determining that the second destination extent for receiving the garbage collected data is full, flushing the valid data copied from the cache memory through the cache memory to the current destination extent, where the flushing of the valid data copied from the cache memory through the cache memory to the current destination extent is flushed in the same order as the destination extent was determined to be full.

[0008] Various embodiments of the garbage collection method may further include, in response to determining that the current source extent to be garbage collected does not contain any portion of valid data that has not been copied to the destination extent, selecting a second source extent as the current source extent to be garbage collected.

[0009] Various embodiments of the garbage collection method include controlling the release of physical address space after flushing of the destination extent cache using a sequence number assigned to each destination extent, each sequence number enabling a lookup of a source extent for which garbage collection has been completed by copying data to that destination extent, the sequence number of the oldest destination extent for which flushing has not been completed being stored as a low watermark sequence number, the sequence number of the current destination extent allocated for garbage collection from the source extent being stored as a high watermark sequence number, the low watermark sequence number being incremented upon completion of flushing of the oldest destination extent from the cache memory, and the high watermark sequence number being incremented as further destination extents are selected for garbage collection.

[0010] Various embodiments of the garbage collection method include, in response to determining that the available virtual address space is less than a threshold, pausing the copying of valid data from the source extent to the destination extent and releasing for available writes physical address space mapped to the virtual address space of at least one destination extent having sequence numbers between values ​​of a low water mark sequence number and a high water mark sequence number.

[0011] Various embodiments of the garbage collection method may include maintaining a journal that stores recovery information for destination extents having sequence numbers between values ​​of a low watermark sequence number and a high watermark sequence number, the recovery information including a mapping between source extents and corresponding destination extents whose valid data was subject to garbage collection.

[0012] Various embodiments of the garbage collection method may further include, in response to determining a loss of cache contents during garbage collection, using the recovery information to reconstruct a forward lookup metadata structure that links a virtual address space to a physical address space for a source extent identified in the recovery information.

[0013] Various embodiments are directed to a computer program product for garbage collection in a storage system in which a virtual address space is mapped to a physical address space, where the virtual address space is addressed in extents, and where the storage system has one or more processors, multiple storage devices, and a cache memory.

[0014] Various embodiments relate to a storage system having a virtual address space mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the storage system including a storage allocator and a garbage collector.

[0015] Various embodiments can protect a data storage system against data loss related to a garbage collection that was in progress at the time of a cache failure. Various embodiments provide a garbage collection algorithm that is coordinated with a cache flush operation related to the garbage collected data. The garbage collection is performed such that old data (i.e., live data in a cache flushed block) is not discarded until the storage allocator is confident that the live data being moved from the garbage collected block has been successfully flushed through the cache to its new physical memory location and is therefore safely arrived at and stored in the destination block for the garbage collected live data.

[0016] Note that while various embodiments are designed to operate successfully in storage systems that use compression, the embodiments may also be used in storage systems that do not use compression. Implementations of the present embodiments may be used for any data reduction that writes new data to a new location and relies on garbage collection. For example, uncompressed data may be written as proposed even if an overwrite to the same location would be successful.

[0017] The freeing of physical address space after flushing of a destination extent cache may be controlled in certain embodiments by assigning a sequence number to each destination extent, each sequence number enabling a lookup of a source extent for which garbage collection has been completed by copying data to that destination extent, the sequence number of the oldest destination extent for which flushing has not been completed being stored as a low watermark sequence number, and the sequence number of the current destination extent allocated for garbage collection from the source extent being stored as a high watermark sequence number, where the low watermark sequence number is incremented when flushing of the oldest destination extent from the cache memory is completed and the high watermark sequence number is incremented as more destination extents are selected for garbage collection. If a shortage of available virtual address space is determined, this may be managed by pausing the copying of valid data from source extents to destination extents and releasing available for writing to physical address space mapped to the virtual address space of at least one destination extent having sequence numbers between the low water mark sequence number and the high water mark sequence number.

[0018] For recovery from accidental loss or corruption of the contents of the cache, it is advantageous to maintain a journal that stores recovery information for each sequence number between a low watermark sequence number and a high watermark sequence number, the recovery information including, for each sequence number, a mapping to a source extent whose garbage collection is completed by moving data to a destination extent associated with said sequence number. During garbage collection, if the contents of the cache are lost, the recovery information may be used to reconstruct forward lookup metadata structures that link virtual address spaces to physical address spaces for the source extents identified in the recovery information.

[0019] The method and storage system are particularly suited, but not limited to, for use in conjunction with an operating system in which each write is performed to a new extent.The method and storage system are particularly suited, but not limited to, for use with operating systems in which data is stored in compressed form.

[0020] In summary, various embodiments provide a garbage collection method for a virtualized storage system in which a virtual address space is addressed by blocks referenced as extents. Old extents are garbage collected by transferring their still valid data to new extents referenced as destination extents. The garbage collected extents may then be freed for reuse by new writes. To account for possible cache losses during the process in which the garbage collected data is written to physical memory, the garbage collected extents are initially withheld from being freed after their garbage collection. Once a destination extent receiving the garbage collected data becomes full, a cache flush of the destination extent is initiated. The memory space of the garbage collected extents associated with the destination extent is then freed conditional on completion of the cache flush.

[0021] The invention will now be further described, by way of example only, with reference to exemplary embodiments illustrated in the drawings, in which: [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 illustrates a computer system environment with a mass storage device. [Diagram 2] FIG. 2 is a block diagram showing the hardware structure of the data storage system. [Figure 3A] FIG. 1 is a flow diagram illustrating steps in a method according to the present disclosure. [Figure 3B] FIG. 1 is a flow diagram illustrating steps in a method according to the present disclosure. [Figure 4] 1A-1D are schematic block diagrams illustrating the progress of garbage collection according to the method disclosed herein. [Diagram 5] FIG. 1 illustrates a cloud computing environment according to an embodiment of the present disclosure. [Figure 6] FIG. 2 illustrates an abstraction model layer according to an embodiment of the present disclosure. [Figure 7] 1 is a schematic diagram of a server connected to a storage system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] In the following detailed description, for purposes of explanation and not limitation, specific details are set forth in order to provide a better understanding of the present disclosure. It will be apparent to those skilled in the art that the present disclosure may be practiced in other embodiments that depart from these specific details.

[0024] As mentioned above, a common strategy for performing writes in a data storage system is to always write to a new data range in the data domain. This strategy has the advantage that it is possible to update a RAID (Redundant Array of Independent Disks) array without requiring a read from the disk. However, as a result of always performing writes to a new data range, a lot of memory space is allocated, which needs to be reclaimed by garbage collection. In fact, if data compression is used, which is optional in some operating systems such as IBM® z / OS®, the size of the write will necessarily vary as its size depends on the compressibility of the data. For example, one write may be worth 8K of space, while another may be only 1K. It is therefore difficult to consistently overwrite data to the right location, since it cannot be guaranteed that the overwritten data will fit into the old memory location when compression is used, and therefore writes must always be performed to new data ranges.

[0025] Garbage collection reclaims old data ranges that, as a result of overwriting, are dominated by obsolete data and contain only sparse live data. Garbage collection proceeds by salvaging the remaining live data and writing it to new data ranges in contiguous blocks. When data is overwritten, it is said to be invalidated. Garbage collection takes the region on the data domain with the invalidated data, moves the live data out of it, and writes it to a new location on the data domain, updating the forward lookup metadata in the process. The forward lookup metadata structure may be implemented as a tree, most commonly a b-tree. Forward b-tree structures map virtual memory addresses to physical memory addresses in virtual storage systems. The reverse lookup metadata describes which volume refers to a particular range of the physical domain, and therefore documents which data subranges are allocated and unallocated on the physical domain. The reverse lookup metadata is then required by the garbage collection algorithm.

[0026] A journal is maintained that can be used to reconstruct the forward lookup metadata structure in the event of a cache failure. A cache failure typically results in the loss of many gigabytes of live data. Journaling ensures that all updates to the forward lookup metadata are logged sequentially. This log may be replayed to reconstruct the forward lookup metadata. Integrating periodic copies (sometimes referred to as "snapshots") of the forward lookup metadata into this scheme removes the need for a potentially constantly growing journal size, since the journal only needs to track forward lookup updates since the last snapshot. If the forward lookup metadata has a tree structure, such as a b-tree, the periodic copying may be referred to as a tree snapshot. Note that the data that the journal references is itself typically not journaled due to the high write bandwidth cost of doing so. Approaches that journal the forward lookup metadata (e.g., b-tree updates) require much less bandwidth. The journaling and updating of the lookup structures may be performed by various modules or components in different embodiments. In various embodiments, the journaling and updating of the lookup structures may be performed by a storage allocator or by a garbage collector.

[0027] A typical "write" is performed as follows: An input-output (IO) is received for a given volume with an address, size and some data. The storage allocator will assign that volume to a storage location on the physical domain. The IO path then writes to that location on the physical domain. Once the write is complete, the forward and reverse lookup structures are updated and the IO is complete. The journal is also updated as part of the forward lookup update.

[0028] A typical "read" would look at the forward lookup structure and identify where on the data domain the data is located. The data domain is then read and the data is returned to the user.

[0029] In case of system failure, cache contents may be lost: If there is a cache below a component that implements data reduction techniques, then in-flight data may be lost (data in flight is any data that has not yet been completely written to the media and would therefore potentially be lost if the cache were discarded). This is a common failure pattern in storage systems and can to some extent be tolerated or at least expected. This is the performance trade-off that is made when using such a cache.

[0030] In storage systems with fixed physical domain locations for updates or thick volumes, cache loss may be tolerable since it only results in stale data being returned if the update was pending in cache at the time of the failure. However, in data-reduced volumes, it is difficult to write consistently to the same location during an overwrite, which means that data that was garbage collected at the time of the failure may also be lost, and this portion of the stale data may not be returned. In file systems, returning stale data is often preferable to returning a media error (data loss).

[0031] FIG. 1 illustrates an exemplary computer system 10. The computer system 10 includes a central processing unit (CPU) 12, which is connected to a mass storage device 14 and a memory device 16. The mass storage device may include a hard disk drive (HDD) device, which may be configured in a Redundant Array of Independent Disks (RAID). Garbage collection operations may be performed on device(s) 14 located in the system 10 or elsewhere. The memory device 16 may include an Electrically Erasable Programmable Read Only Memory (EEPROM) or a host of related devices, such as RAM, ROM, FLASH, etc. The memory device 16 and the mass storage device 14 are connected to the CPU 12 via a signal transmission medium. Additionally, the CPU 12 is connected via a communication port 18 to a communication network 20 having an additional plurality of attached computer systems 22 and 24. Computer system 10 may include one or more processor devices (e.g., CPU 12) and additional memory devices 16 for each individual component of computer system 10 to perform and implement garbage collection and other operations as described herein.

[0032] FIG. 2 is an exemplary block diagram 200 illustrating a hardware structure of a data storage system in a computer system for implementing an embodiment of the present invention. The host computers 210, 220, and 225 each function as a central processing unit for performing data processing as part of the data storage system 200. The hosts (physical or virtual devices) 210, 220, and 225 may be one or more new physical or logical devices for achieving the objectives of the present invention in the data storage system 200. In one embodiment, by way of example only, the data storage system 200 may be implemented as an IBM® System Storage® V9000®. The network connection 260 may be a Fibre Channel fabric, a Fibre Channel point-to-point link, a Fibre Channel over Ethernet fabric or point-to-point link, a FICON or ESCON I / O interface, any other I / O interface type, a wireless network, a wired network, a LAN, a WAN, heterogeneous or homogeneous, public (i.e., Internet) or private, or any combination thereof. The hosts 210, 220, 225 may be local or distributed among one or more locations and may include any type of fabric (or fabric channel) or network adapter 260, such as Fibre Channel, FICON, ESCON, Ethernet, Fiber Optic, Wireless, or Coaxial adapters, to the storage controller 240. The data storage system 200 thus includes an appropriate fabric or network adapter 260 for communicating. The data storage system 200 is illustrated in FIG. 2, including a storage controller 240 and a storage 230.

[0033] 2 as a single processing unit including a microprocessor 242, a system memory 243, and non-volatile storage (NVS) 216. The storage controller 240 may, however, include multiple processing units, each with its own processor complex and system memory, interconnected by a dedicated network within the data storage system 200. The storage 230 may be comprised of one or more storage devices, such as a storage array, connected to the storage controller 240 by a storage network.

[0034] In some embodiments, the devices included in storage 230 may be connected in a loop architecture. Storage controller 240 manages storage 230 and facilitates processing of write and read requests intended for storage 230. System memory 243 of storage controller 240 stores operational software 250, program instructions and data that processor 242 can access to perform functions and method steps associated with managing storage 230, including garbage collection. As shown in FIG. 2, system memory 243 includes an internal memory cache 245 that is a cache for storage 230 (or communicates with an external cache). The cache memory is used to buffer write and read data for write and read requests, respectively. In one embodiment, cache 245 is allocated to a device external to system memory 243, but remains accessible by microprocessor 242 and can function to provide additional security against data loss.

[0035] In some embodiments, the cache 245 is implemented with volatile and non-volatile memory and is coupled to the microprocessor 242 via a local bus (not shown in FIG. 2) to improve performance of the data storage system 200. The NVS 216 included in the data storage controller 240 is accessible by the microprocessor 242 and provides additional support for operations including garbage collection. The NVS 216 may be implemented with non-volatile memory that does not require power to retain data and may therefore be referred to as a persistent cache. The NVS 216 may be integrated into the cache 245. In some embodiments, a backup power source (not shown in FIG. 2), such as a battery, provides the NVS 216 with enough power to retain the data stored therein in the event of a loss of power to the data storage system 200. In certain embodiments, the capacity of the NVS 216 is equal to or less than the total capacity of the cache 245.

[0036] Storage 230 may be physically composed of one or more storage devices, such as a storage array. A storage array is a logical grouping of individual storage devices, such as hard disks. In some embodiments, storage 230 is composed of a Just a Bunch of Disks (JBOD) array or a RAID array. A collection of physical storage arrays may be further combined to form ranks, which separate the physical storage from the logical configuration. Storage space within a rank may be allocated to logical volumes, which define the storage locations specified in write / read requests.

[0037] As shown in FIG. 2, a storage system may include logical volumes or simply "volumes" and may have different types of allocations. Storages 230a, 230b, and 230n are shown as ranks in the data storage system 200 and are referred to herein as ranks 230a, 230b, and 230n. The ranks may be local to the data storage system 200 or may be located at physically separate locations. In other words, a local storage controller may connect with a remote storage controller and manage storage at the separate locations. Rank 230a is shown configured with two whole volumes 234, 236 and one partial volume 232a. Rank 230b is shown with another partial volume 232b. Thus, volume 232 is allocated across ranks 230a and 230b. Rank 230n is shown as being assigned in its entirety to volume 238, i.e., rank 230n references the entire physical storage for volume 238. From the above examples, it will be appreciated that a rank may be configured to include one or more partial volumes or an entire volume, or both.

[0038] The storage controller 240 includes a garbage collection module 255 that assists in garbage collection in a journaled file system. The garbage collection module 255 operates in conjunction with the components of the storage controller 240, the hosts 210, 220, 225, and the storage device 230. The garbage collection module 255 may be located within the cache 245 or other components of the storage controller 240.

[0039] The storage controller 240 includes a control switch 241 that controls the Fibre Channel protocol to the host computers 210, 220, 225, a microprocessor 242 that controls the entire storage controller 240, a non-volatile control memory 243 that stores a microprogram (operation software) 250 for controlling the operation of the storage controller 240, a cache 245 that temporarily stores (buffers) data, a buffer 244 that assists the cache 245 to read and write data, the control switch 241 that controls the protocol that controls the transfer of data to and from the storage device 230, and a garbage collection module 255 in which information can be set. Multiple buffers 244 may be implemented to assist garbage collection in a journaled file system.

[0040] The host computer or one or more physical or virtual devices 210, 220 and 255 and the storage controller 240 are connected through a network adapter (which may be a Fibre Channel) 260 as an interface, i.e., via a switch referred to as a "fabric". The microprocessor 242 controls the memory 243, which can store command information from the host device (physical or virtual) 210 and information for identifying the host device (physical or virtual) 210. The control switch 241, the buffer 244, the cache 245, the operation software 250, the microprocessor 242, the memory 243, the NVS 216 and the garbage collection module 255 communicate with each other. Some or all of the components, such as the operation software 245, may be incorporated in the memory 243 to perform garbage collection in a journal file system. Each component in the illustrated device may be coupled together and may communicate with each other.

[0041] FIG. 7 illustrates a server 710 connected to a storage system 712 in which the principles of the present invention may be implemented in various embodiments. The server 710 performs read and write operations to the storage system 712 via an interface, such as an interface using a standard such as SCSI (Small Computer system interface). The storage system 712 has a number of connected physical components that provide processing and storage functionality. Primary long-term storage functionality is provided by a set of physical storage devices 714 (preferably consisting of a set of disks), which are divided into logical allocation areas 716. Additional memory is also provided within the storage system 712 by a write cache 718, which is used locally by the processing components of the storage system 712.

[0042] From the perspective of the external server 710, the storage system 712 presents one or more virtual disks 720 (vdisks) that logically present the stored data to the external server 710. Data represented to the server 710 as residing on a single vdisk 720 may actually be stored on different disks 714 within the physical storage area 714. The mapping between the vdisks 720 and the physical disks 714 is managed by two components within the storage system 712: a forward lookup structure 722 and a reverse lookup structure 724. The forward lookup structure 722 maps one or more addresses in the vdisk 720 to the actual physical location(s) within the disks 714 where the data is stored. The reverse lookup structure 724 maps the physical location(s) within the disks 714 back to an address(es) within one or more vDisks 720.

[0043] When a read arrives at vdisk 720 from server 710, vdisk 720 communicates with forward lookup structure 722 to obtain the physical address of the data and reads the required data from the address in physical domain 714. In the case of a read, no changes are required to either the forward lookup structure 722 or the reverse lookup structure 724.

[0044] The behavior is more complicated when a write arrives from server 710 to vdisk 720. When new data is written to disk 714, forward lookup structure 722 must be updated to connect the address of the new data in vdisk 720 to the physical location of the data as it was actually stored in disk 714. In this case, both forward lookup structure 722 and reverse lookup structure 724 must be updated to reflect the connection between the data presented to server 710 by vdisk 720 and the actual physical location on disk 714 where the data is stored.

[0045] The storage system 712 also includes two further components, a storage allocator 726 and a garbage collector 728. The storage allocator 726 performs several functions, firstly, allocating live blocks from live regions 716 in the physical domain 714 that are communicated to the vdisk 720, and maintaining a free list for allocated data, which is described in more detail below. When new data is written into the storage system 712, the forward lookup structure 722 writes to a location determined by the allocated block location. The top-level storage allocator 726 has a set of "active" regions 716 that are tracked at any given time. A finite number of these regions 716 may be tracked. For example, in an exabyte physical domain 714 with a region size of (say) 1 gigabyte, this would result in tracking over 1 billion regions. Even if a region entry in the storage allocator 726 only cost 4B (i.e., to track how many blocks in a region 716 were available), this would consume 4 gigabytes of memory that would be better used as a cache. Thus, a set of active regions 716 is maintained up to a size of N regions 716, where N is the maximum number that can be handled by the cache 718. When all the storage in an active region 716 is exhausted, this entire region 716 leaves the set of active regions 716 and the memory used to track the regions 716 may be reclaimed and, if applicable, garbage collected by the garbage collector 728. When all N regions 716 are active, no further regions 716 can be allocated, which means that the free list (a list of unallocated blocks of memory) cannot receive additional host blocks. The garbage collector 728 essentially reclaims the regions 716 with the limited live data that exists by rewriting the remaining live data to new regions 716, thereby releasing these reclaimed regions 716 to the storage allocator 726.A garbage collector 728 (or garbage collection module) connects to the backward lookup structure 724 that is used to perform garbage collection.

[0046] The storage system 712 is presented to the server 710 as thin-provisioned (also known as space-efficient), where the underlying storage can be much smaller than that presented to the host (server 710). This is achieved by metadata structures that map virtual domain writes to the physical domain, and by compression and deduplication. Implementations that utilize compression must implement a mechanism for space recovery, known as garbage collection, because there is no guarantee that any two writes to the same virtual address will use the same amount of physical storage. Over time, the disk 714 is left with many small regions of free space, and it is desirable to coalesce these by moving writes in the physical domain (and updating their mapping in the virtual domain).

[0047] Next, the role of the journal is explained. The journal tracks updates into a b-tree, which stores mappings between virtual and physical addresses. The role of the journal is to track changes to these mappings in case an error occurs during the update. The b-tree may then be restored by reference to recovery information in the journal. Corruption of the mappings between the virtual and physical domains as a result of loss of cache contents or other errors is avoided.

[0048] In the context of this overwrite operation, the steps involved in logging it to the journal are writing the new data to the new location, updating the b-tree to log the physical address of the new location, and marking the old physical data as invalid (stale). Garbage collection must then be performed to move the non-invalid data (i.e., data that is still valid) that is intermixed with the invalid data. Typically, the valid data in a particular virtual memory block will make up only a very small portion of the total, say 10% or less. Garbage collection will move the still valid data that may be the coldest (i.e., on main memory, on physical storage, no longer in cache memory), and the work of garbage collection will cause this cold data to become hot, i.e., in-flight in cache memory, and therefore at risk of loss in the event of a crash.

[0049] Various embodiments provide a method of garbage collection for a virtualized storage system in which a virtual address space is mapped to a physical address space. The method may be implemented by modifying the storage allocator of the system. The method may be used in storage systems that store data in a compressed format, such as IBM® z / OS, as well as in uncompressed storage systems. The method is particularly intended for storage systems that constantly write to new virtual memory space to reduce reads, thus consuming a lot of virtual memory space and requiring ongoing garbage collection. The storage allocator, according to various embodiments, tracks the virtual memory used for writes on a block-by-block basis, where each block is a range of virtual memory addresses. These blocks are called "extents."

[0050] In various embodiments, the garbage collection method proceeds as follows: The storage allocator selects an extent to receive the garbage collected data, which is referred to as the destination extent or target extent.

[0051] The storage allocator selects extents to be garbage collected, referred to as source extents, and the virtual addresses of the source extents are tagged as containing either valid or invalid data. As mentioned above, the amount of valid data is likely to be very small with respect to the size of the source extents, i.e., much smaller than the amount of invalid data. In other words, each destination extent is likely to have enough space to receive all the valid data from several source extents, of which 5-10 could be a typical number.

[0052] The storage allocator, or in some embodiments the garbage collector 728 (or garbage collection module 255), moves or copies valid data in the source extent to the destination extent via the cache memory, then iteratively selects the second, third source extent, and so on, for garbage collection, each time moving valid data to the destination extent via the cache memory. This proceeds until the destination extent is full. The garbage collector 728 (or garbage collection module 255) may determine when the destination extent is full. When the destination extent is full, the remaining data that is still to be moved or copied from the source extent being garbage collected may be rolled, i.e., copied to the next allocated destination extent. The entirety of the source extent data from a given source extent does not have to fit into one destination extent. This may be split across two, and since various embodiments flush in order, it only needs to be tagged as dependent on the last one, so that both destination extents can be guaranteed to be flushed.

[0053] When a point is reached where the destination extent currently being used for garbage collection is full, the storage allocator, or garbage collector, selects another extent to be the next destination extent, thereby allowing for continued garbage collection. The process then repeats by filling the next destination extent with valid garbage collected data, and so on.

[0054] Asynchronously with the ongoing filling of the destination extent sequence and the emptying of any number of source extents of still valid data, each full destination extent is flushed from the cache in turn. This flushing of each destination extent from the cache memory begins only when each such destination extent becomes full, in the order in which they were allocated and filled. The flushing and garbage collection transfers are independently controlled, so the amount of time it takes each destination extent to be flushed may be slower or faster relative to the garbage collection speed. Because of the asynchronous nature between the flushing and garbage collection transfers between the source and destination extents, the virtual address space of the garbage collected source extent is not made immediately available for new writes, but is held temporarily. Freeing is done first after each destination extent flush is completed, where "freeing" means making available for writes the physical address space that is mapped into the virtual address space of the source extent whose valid data has been completely flushed out of the cache memory by the flushing of that particular destination extent. Thus, although the cache flushing and subsequent freeing of stale addresses is not performed synchronously with the garbage collection from the source extent to the destination extent, these activities are coordinated.

[0055] FIG. 3A shows the process flow of garbage collection activity. In step S1, a destination extent is selected to receive garbage collected data. In step S2, a source extent is selected for garbage collection. Steps S1 and S2 may be performed by a storage allocator. In step S3, valid data in the source extent is identified, and in step S4, this valid data is moved or copied to the destination extent. Steps S4 and S5 may be performed by a storage allocator or garbage collector. In step S5, if the destination extent is full (as determined by the storage allocator or garbage collector), the process flow returns to step S1 to select a new destination extent for continued garbage collection, and the remaining data from the existing source extent or a new source extent is selected in step S2. The now full destination extent is a candidate for cache flushing as shown in FIG. 3B.

[0056] FIG. 3B shows the process flow of cache flushing and freeing of garbage collected extents, i.e., source extents. In step F1, a full destination extent is selected for cache flushing. In step F2, cache flushing may be initiated by the garbage collector. In step F3, once cache flushing is completed as determined in step F3 by the garbage collector, in step F4, the memory space mapped to the source extent that was just garbage collected into the destination extent that was just flushed may be freed for new writes by the storage allocator.

[0057] More specifically, the method may be implemented such that every write to disk has a metadata header that documents which volume and forward lookup logical block address (LBA) it originally belonged to. Once all metadata updates are complete, the backward lookup structures, such as b-trees, associated with a particular extent are updated and flushed to main memory (cache to disk) via the cache memory. Once garbage collection is complete, the garbage collected extent has its backward lookup metadata updated to document that it has been garbage collected, but the garbage collected extent is not immediately disposed of, i.e., made available for new writes immediately, but rather is temporarily retained and remains available even if the contents of the cache are lost. The backward lookup metadata for a garbage collected extent is discarded only if the extent is subsequently reused or discarded.

[0058] Extents may be used by implementing multiple forward lookup structures at once, e.g. b-trees. When all metadata updates for an extent are complete, a cache flush is triggered. Since writes are always written to a "new location" (either to newly allocated storage, or to a previously garbage-collected extent), the cache effectively only acts as a temporary buffer for full-stride writes.

[0059] The storage allocator distinguishes between current extents being used to receive new writes inbound to the system and current extents being used internally to receive garbage collected data from other extents. The storage allocator tracks the set of extents that are currently active in garbage collection through the use of a pair of sequence numbers (in a system that supports multiple storage nodes, a separate sequence number can be used for each node).

[0060] When garbage collection is performed on an extent, rather than making the old extent available for reuse immediately after it has been garbage collected, the old extent is temporarily held, i.e., its availability as a new extent for writes is delayed. Because live data from a garbage collected extent may be allocated to be moved to multiple destination extents, and tracking each of these is expensive, various embodiments track the extent currently being used for writes through a pair of sequence numbers, referred to herein as a low watermark sequence number and a high watermark sequence number. The use of low watermark sequence numbers and high watermark sequence numbers simplifies the control of the release of physical address space after a cache flush of a destination extent. The storage controller assigns a unique sequence number to each destination extent. Each sequence number allows for a lookup of a source extent that has completed its garbage collection by moving data to that destination extent. The sequence number of the oldest destination extent that has not completed flushing is stored as the low watermark sequence number. The sequence number of the destination extent currently allocated for garbage collection from the source extent is stored as the high water mark sequence number. The value of the low water mark sequence number is incremented when flushing of the oldest destination extent from cache memory is completed, and the value of the high water mark sequence number is incremented when an additional destination extent is selected for garbage collection. Because the rate of flushing is variable and may be slower than the destination extents are filled over some periods and faster over other periods, the difference between the values ​​of the low water mark sequence number and the high water mark sequence number will increase and decrease over time.

[0061] Because destination extents are flushed in order of increasing sequence number, the low watermark sequence number always corresponds to the oldest destination extent that has not yet been flushed. The range of extents with indexes between the low watermark sequence number and the high watermark sequence number indicate those destination extents (and thus the garbage-collected extents that have received garbage-collected data from them) that are at potential risk from cache content loss or other cache errors. It is safe to release for new writes any garbage-collected extents that depend on a destination extent only if it is known with certainty that the destination extent has been flushed to disk.

[0062] The presence of low watermark sequence numbers and high watermark sequence numbers also simplifies what needs to be stored in the journal to protect against cache errors. The storage controller can limit the recovery information stored in the journal to memory addresses linked to those associated with sequence numbers between the low watermark and high watermark values. The recovery information for each such sequence number includes a mapping to a source extent whose garbage collection was completed by moving data to a destination extent associated with the sequence number. If the contents of the cache are lost during garbage collection, the recovery information is used to reconstruct the forward lookup metadata structures that link the virtual address space to the physical address space for the source extents identified in the recovery information. That is, the journal can attempt to recover the forward lookup structures through a replay approach. This replay approach involves reading a journal entry that indicates the physical address of the destination extent, then reading the destination extent in the physical domain, and determining whether the location referenced by the journal entry has the correct metadata header that belongs to the volume being recovered. If a mismatch is found between the journal and data domains, journal replay continues with later journal entries and the address being recovered is left as a media error and is subject to later journal entries that find more valid data. This replay approach attempts to find the last reference that has not been overwritten. Further details of this replay approach are disclosed in commonly owned, co-pending U.S. patent application Ser. No. 16 / 548,474, filed Aug. 22, 2019, entitled “Data Validity Verification During Data Recovery in a Log-Structured Array Storage System,” which is incorporated by reference in its entirety. If it is identified that a reference to a garbage collected extent has been recovered, the extent in question should be treated as if garbage collection was not successful and therefore requires a second round of garbage collection.Since the reverse lookup metadata can mark restored references as invalid, it is necessary to mark the extents for garbage collection without using the reverse lookup structures. This can be achieved by utilizing the header data associated with each write. Garbage collection will inject all volume writes and allow the header to be identified into the IO path. The IO path will reject writes that come in from a location that does not originally match the forward lookup structures. This is done so that it removes the requirement that the reverse lookup structures match the forward lookup structures exactly, for garbage collections with and without reverse lookup structures. This reclamation approach will always find data that belongs to garbage collected extents in the event of a cache loss, since it identifies and prevents the reuse of extents that belong to garbage collected extents before flushing is complete.

[0063] The proposed method will result in consuming more memory space than the conventional method, since it holds on to the extents longer than the approach of immediately reusing the garbage collected extents. Therefore, keeping in mind that a particular user, business entity or application will typically only allocate a fixed amount of virtual memory space, using the proposed method may result in a shortage of virtual memory space. In particular, if flushing slows down for any reason, the difference between the low water mark and the high water mark may become large, which means that a large amount of memory space may be tied up for garbage collection. As a result, when the amount of free virtual memory falls below a certain threshold, for example, below 10% or 5% of the total available virtual memory, it will be beneficial to stop garbage collection activity according to the proposed method, thereby allowing the virtual memory space allocated to the garbage collection destination extents for those destination extents for which the source extents for which have not yet been released to be released to be made available for new writes. If the storage allocator determines that there is insufficient available virtual address space, it suspends the garbage collection by suspending the movement of valid data from source extents to destination extents. It can then free up the physical address space that is mapped to the virtual address space of destination extents with sequence numbers between the low and high water marks to be available for writes. This release is not actually instantaneous, since it first notifies all parts of the storage allocator to stop using these destination extents. Once all the storage associated with these destination extents has been reported as fully used or discarded, these destination extents can be freed up for new writes. Another option to handle the same problem is to cap the difference between the low and high water marks, imposing a maximum number of destination extents that can be attached to a garbage collection.When this cap is hit, garbage collection will hold off until the low water mark sequence number is incremented, i.e. until the current cache flush is completed.

[0064] 4(A)-4(D) are schematic block diagrams showing an example of the progression of garbage collection according to the proposed method. A timeline is shown in schematic form from FIG. 4(A) to FIG. 4(D). Initially, the virtual memory is using three exemplary extents Ext1, Ext2, and Ext3, with transfer to physical storage occurring via cache. Extent 1 (Ext1) contains a mix of cold data and stale (still valid) data to be garbage collected. Extent 2 (Ext2) contains static cold data. Extent 3 (Ext3) is the extent currently being used for new writes. Extent 1 is then identified as a candidate for garbage collection based on the relatively small percentage of still valid data. The garbage collection process reads the extents and selects the data to keep (401). As shown diagrammatically in Figure 4(B), the stale data from Extent 1 is identified for discarding and the still valid "live" data from Extent 1 is written to Extent 3. Figure 4(C) shows that even after garbage collection from Extent 1 to Extent 3 is complete, the data is retained in Extent 1 while the garbage collected data can potentially migrate through the cache to physical storage. Figure 4(D) shows how Extent 1 is discarded with respect to Extent 3 once the cache has been flushed to the physical storage device.

[0065] It will be apparent to those skilled in the art that all or a portion of the logical processing steps of the preferred embodiments may alternatively be embodied in a logical device or multiple logical devices that include logical elements configured to perform the logical processing steps of the method, and that such logical elements may include hardware components, firmware components, or a combination thereof.

[0066] It will also be apparent to those skilled in the art that all or part of the logic components of the preferred embodiments may alternatively be embodied in a logic device comprised of logic elements to perform the steps of the method, and such logic elements may include components such as logic gates in, for example, a programmable logic array or an application specific integrated circuit. Such logic arrangements are further embodied in enabling the elements to temporarily or permanently establish logic structures in the array or circuit, for example using a virtual hardware description language, which may be stored or transmitted using a fixed or transmissible carrier medium.

[0067] In yet another embodiment, the present invention may be realized in the form of a computer-implemented method of deploying a service, comprising the step of deploying a computer program operable to be deployed on a computer infrastructure and, when executed thereon, cause a computing device to perform all of the steps of the method.

[0068] It will be appreciated that the methods and components of the preferred embodiments may alternatively be embodied in whole or in part in a parallel computing system including two or more processors for executing parallel software.

[0069] Further embodiments of the present disclosure are computer program products defined with respect to systems and methods, which may include computer readable storage medium(s) having computer readable program instructions for causing a processor to implement aspects of the present invention.

[0070] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device.

[0071] The present invention may be a system, method or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0072] A computer readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or ridge structures in grooves having instructions recorded thereon, and any suitable combination of the above. Computer-readable storage media, as used herein, is not to be construed as a transitory signal per se, such as an electric wave, a freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0073] The computer readable program instructions described herein may be downloaded into each computing / processing device from a computer readable storage medium or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in the computer readable storage medium within the respective computing / processing device.

[0074] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object oriented languages ​​such as Smalltalk®, C++ or the like, procedural languages ​​such as the C programming language or similar programming languages. The computer readable program instructions may be executed as a stand-alone software package, entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, the electrical circuitry may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to individualize the electrical circuitry, which may include, for example, programmable logic circuitry, a field programmable gate array (FPGA), or a programmable logic array (PLA), to perform aspects of the invention.

[0075] Aspects of the disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer readable program instructions that, when provided to a computer processor or other programmable data processing device, produce a machine such that the instructions, executed via the computer processor or other programmable data processing device, create means for implementing the functions / acts identified in a block or blocks of the flowchart illustrations and / or block diagrams.

[0076] These computer readable program instructions may also be stored on a computer readable storage medium that can instruct a computer, programmable data processing device or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium having instructions stored thereon includes an article of manufacture including instructions that implement an aspect of the functionality / action identified in a block or blocks of the flowchart illustrations and / or block diagrams.

[0077] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps to generate a computer implemented process such that the instructions executing on the computer, other programmable data processing apparatus, or other device implement aspects of the functionality / actions identified in a block or blocks of the flowchart and / or blocks.

[0078] The flowchart diagrams and block diagrams in the drawings illustrate the architecture, functionality and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart diagrams or block diagrams may represent a module, segment or portion of instructions, including one or more executable instructions for implementing a particular logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order depicted in the drawings. For example, two blocks shown in succession may in fact be realized in one step and executed simultaneously, substantially simultaneously, partially or fully overlapping in time, or the blocks may be executed in reverse order depending on the functionality involved. It should be noted that each block of the block diagrams and / or flowchart diagrams and combinations of blocks in the block diagrams and / or flowchart diagrams may be implemented by a special purpose hardware-based system that performs a particular function or action or executes a combination of special purpose hardware and computer instructions.

[0079] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings detailed herein is not limited to a cloud computing environment. Rather, embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or hereafter developed.

[0080] Cloud computing is a model of service delivery that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0081] The characteristics are as follows:

[0082] On-Demand Self-Service: Cloud consumers can unilaterally provision computing power, such as server time and network storage, as they need it automatically without the need for human interaction with the service provider.

[0083] Broadband Network Access: Capabilities are available over the network and accessed via standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, PDAs).

[0084] Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where various physical and virtual resources are dynamically allocated and reallocated according to demand. Consumers generally have no control or knowledge of the exact location of the resources provided to them, but there is a sense of location independence in the sense that they can specify location at a higher level of abstraction (e.g. country, state, or data center).

[0085] Rapid Elasticity: Capacity can be rapidly and elastically, in some cases automatically, provisioned to quickly scale out and rapidly released to quickly scale in. To the consumer, the provisioned available capacity often appears externally unlimited, and any amount can be purchased at any time.

[0086] Metered Services: Cloud systems automatically control and optimize resource usage by utilizing metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active users). Resource usage is monitored, controlled, and reported, providing transparency to both providers and consumers of the services being used.

[0087] The service model is as follows:

[0088] Software as a Service (SaaS): The capability offered to the consumer is to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities with the potential exception of limited user-specific application configuration settings.

[0089] Platform as a Service (PaaS): The capability offered to a consumer is to deploy consumer-created or acquired applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems or storage, but does have control over the deployed applications and possibly the configuration of the application hosting environment.

[0090] Infrastructure as a Service (IaaS): The capability provided to the consumer is to provide processing, storage, networking, and other basic computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and in some cases, limited control over selected networking components (e.g., host firewalls).

[0091] The deployment model is as follows:

[0092] Private Cloud: The cloud infrastructure is used exclusively for one organization. It may be managed by the organization or a third party and may be on-premise or off-premise.

[0093] Community Cloud: Cloud infrastructure is shared by several organizations to support a specific community with common concerns (e.g., mission, security requirements, policy and compliance considerations). It may be managed by the organization or a third party and may exist on-premise or off-premise.

[0094] Public Cloud: The cloud infrastructure is available to the general public or large industry entities and is owned by organizations that sell cloud services.

[0095] Hybrid Cloud: A cloud infrastructure is a blend of two or more clouds (private, community or public) that remain unique entities but are joined by standardized or proprietary technologies that allow data and application portability (e.g. cloud bursting for load balancing between clouds).

[0096] Cloud computing environments are service-oriented with an emphasis on statelessness, low coupling, modularity and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0097] Referring now to FIG. 5, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof, may communicate. The nodes 10 may communicate with each other. They may be physically or virtually grouped (not shown) in one or more networks, such as private, community, public, or hybrid clouds, as described above, or combinations thereof. This enables the cloud computing environment 50 to provide infrastructure, platform, or software, or combinations thereof, as a service, for which the cloud consumer does not need to maintain resources on a local computing device. The types of computing devices 54A-54N shown in FIG. 5 are for illustrative purposes only, and it will be understood that the computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network, a network addressable connection (e.g., using a web browser), or both.

[0098] An add-on according to an embodiment of the present disclosure may be installed in a web browser in the environment of Figure 5 as follows: One of the cloud computing nodes 10 may host a website from which an add-on may be downloaded, upon request, to a third party computing device, such as any of computing devices 54A, 54B and 54C. The request causes the add-on to be transmitted from node 10 to computing device 54A / 54B / 54C over a network connection, where the add-on is transmitted along with an installer for integration into a web browser already present on the computing device.

[0099] Referring now to Figure 6, a set of functional abstraction layers provided by cloud computing environment 50 (Figure 5) is shown. It will be understood in advance that the components, layers and functions shown in Figure 6 are for illustrative purposes only and embodiments of the present disclosure are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0100] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframes 61, servers based on RISC (Reduced Instruction Set Computing) architecture 62, servers 63, blade servers 64, storage devices 65, and networks and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0101] The virtualization layer 70 provides an abstraction layer from which examples of virtualized entities such as virtualized servers 71, virtualized storage 72, virtualized networks including virtual private networks 73, virtualized applications and operating systems 74, and virtual clients 75 are provided.

[0102] In one example, the management layer 80 may provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides tracking of costs as resources are utilized within the cloud computing environment and billing or invoicing for consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides allocation and management of cloud computing resources to meet required service levels. Service level agreement (SLA) planning and fulfillment 85 provides pre-allocation and procurement of cloud computing resources in anticipation of future demand in accordance with SLAs.

[0103] The workload layer 90 provides examples of functionality for which a cloud computing environment is utilized. Examples of workloads and functionality provided from this layer include mapping and navigation 91, software lifecycle management 92, virtual classroom instructional delivery 93, data analytics processing 94, transaction processing 95, and a storage allocator 96 with garbage collection management as described above in accordance with embodiments of the present disclosure.

[0104] It will be apparent to those skilled in the art that many modifications and variations can be made to the exemplary embodiments described above without departing from the scope of the present disclosure.

Claims

1. A method of garbage collection in a storage system in which a virtual address space is mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the method comprising: selecting a first destination extent as a current destination extent for receiving the garbage collected data; selecting a first source extent as a current source extent to be garbage collected, the current source extent including one or more portions of valid data; determining whether a destination extent selected as the current destination extent for receiving garbage collected data is full; determining whether the source extent selected as the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to a destination extent; in response to determining that the current destination extent for receiving garbage collected data is not full and determining that the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to a destination extent, copying one of the portions of valid data that have not been copied to a destination extent from the current source extent to the current destination extent via the cache memory; in response to determining that the current destination extent for receiving garbage collected data is full, flushing the valid data from the cache memory through the cache memory to be copied to the current destination extent; after flushing the valid data from the cache memory that has been moved to the current destination extent, releasing it for writing to the physical address space that is mapped to the virtual address space of one or more source extents from which all valid data has been copied to the current destination extent via the cache memory; controlling the freeing of the physical address space after flushing of the destination extent's cache by assigning a sequence number to each destination extent, each sequence number enabling a lookup of the source extent whose garbage collection has been completed by copying data to that destination extent; Including, the sequence number of the oldest destination extent for which flushing has not been performed is stored as a low water mark sequence number; the sequence number of the current destination extent allocated for garbage collection from the source extent is stored as a high water mark sequence number; the low water mark sequence number is incremented upon completion of flushing of the oldest destination extent from the cache memory; The method of claim 1, wherein the high water mark sequence number is incremented as further destination extents are selected for garbage collection.

2. in response to determining that the available virtual address space is less than a threshold, pausing the copying of valid data from the source extent to the destination extent; releasing so as to be available for writing to the physical address space mapped into the virtual address space of at least one destination extent having a sequence number between the values ​​of the low water mark sequence number and the high water mark sequence number; The method of claim 1 , comprising:

3. maintaining a journal that stores recovery information for destination extents having sequence numbers between the values ​​of the low water mark sequence number and the high water mark sequence number, the recovery information including a mapping between source extents and corresponding destination extents whose valid data has been subject to garbage collection. The method of claim 1 or 2, further comprising:

4. responsive to determining a loss of cache contents during garbage collection, using the recovery information to reconstruct a forward lookup metadata structure linking a virtual address space to a physical address space for the source extent identified in the recovery information. The method of claim 3 further comprising:

5. A computer program for garbage collection in a storage system in which a virtual address space is mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the computer program comprising: selecting a first destination extent as a current destination extent for receiving the garbage collected data; selecting a first source extent as a current source extent to be garbage collected, the current source extent including one or more portions of valid data; determining whether a destination extent selected as the current destination extent for receiving garbage collected data is full; determining whether a source extent selected as the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to a destination extent; in response to determining that the current destination extent for receiving garbage collected data is not full and that the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to a destination extent, copying one of the portions of valid data that have not been copied to a destination extent from the current source extent to the current destination extent via the cache memory; in response to determining that the current destination extent for receiving garbage collected data is full, flushing the valid data from the cache memory through the cache memory to be copied to the current destination extent; flushing the valid data from the cache memory that has been moved to the current destination extent, and then releasing it for writing to the physical address space that is mapped to the virtual address space of one or more source extents from which all valid data has been copied to the current destination extent via the cache memory; assigning a sequence number to each destination extent to control the release of the physical address space after flushing of the destination extent's cache; and for each sequence number, a lookup is performed of the source extent whose garbage collection has been completed by copying data to the destination extent; the sequence number of the oldest destination extent for which flushing has not been performed is stored as a low water mark sequence number; the sequence number of the current destination extent allocated for garbage collection from the source extent is stored as a high water mark sequence number; the low water mark sequence number is incremented upon completion of flushing of the oldest destination extent from the cache memory; The high water mark sequence number is incremented when a further destination extent is selected for garbage collection.

6. in response to determining that the available virtual address space is less than a threshold, pausing the copying of valid data from the source extent to the destination extent; releasing so as to be available for writing to the physical address space mapped to the virtual address space of at least one destination extent having a sequence number between the values ​​of the low water mark sequence number and the high water mark sequence number; 6. The computer program product of claim 5, further comprising:

7. 7. The computer program product of claim 5 or 6, further comprising maintaining a journal storing recovery information for destination extents having sequence numbers between values ​​of the low water mark sequence number and the high water mark sequence number, the recovery information including a mapping between source extents and corresponding destination extents whose valid data has been subject to garbage collection.

8. 8. The computer program product of claim 7, further causing the processor to, in response to determining a loss of cache contents during garbage collection, use the recovery information to reconstruct a forward lookup metadata structure that links a virtual address space to a physical address space for the source extent identified in the recovery information.

9. A storage system having a virtual address space mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the storage system comprising: a storage allocator for selecting a first destination extent as a current destination extent for receiving the garbage collected data; the storage allocator for selecting a first source extent as a current source extent to be garbage collected, the current source extent including one or more portions of valid data; a garbage collector that determines whether a destination extent selected as the current destination extent for receiving garbage collected data is full; the garbage collector determining whether a source extent selected as the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to a destination extent; in response to determining that the current destination extent for receiving garbage collected data is not full and that the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to a destination extent, the garbage collector for copying, via the cache memory, one of the portions of valid data that have not been copied to a destination extent from the current source extent to the current destination extent; the garbage collector for flushing the valid data being copied from within the cache memory through the cache memory to the current destination extent in response to determining that the current destination extent for receiving garbage collected data is full; the storage allocator for flushing the valid data from the cache memory that has been moved to the current destination extent, and then releasing the valid data for writing to the physical address space that is mapped to the virtual address space of one or more source extents from which all valid data has been copied to the current destination extent via the cache memory; the storage allocator controlling the release of the physical address space after flushing of the destination extent's cache by assigning a sequence number to each destination extent; each sequence number allows for a lookup of the source extent whose garbage collection has been completed by copying data to that destination extent; the sequence number of the oldest destination extent for which flushing has not been performed is stored as a low water mark sequence number; the sequence number of the current destination extent allocated for garbage collection from the source extent is stored as a high water mark sequence number; the low water mark sequence number is incremented upon completion of flushing of the oldest destination extent from the cache memory; 13. A storage system comprising: a storage unit that stores a garbage collection extent for a first extent of the first extent;

10. in response to determining that the available virtual address space is less than a threshold, the storage allocator suspending copying of valid data from a source extent to a destination extent; the storage allocator releasing for available for writing to the physical address space mapped to the virtual address space at least one destination extent having a sequence number between the values ​​of the low water mark sequence number and the high water mark sequence number; The storage system of claim 9 .

11. 11. The storage system of claim 9 or 10, further comprising: the garbage collector maintaining a journal that stores recovery information for destination extents having sequence numbers between values ​​of the low water mark sequence number and the high water mark sequence number, the recovery information including a mapping between source extents and corresponding destination extents whose valid data has been subject to garbage collection.

12. 12. The storage system of claim 11, further comprising the garbage collector, in response to determining a loss of cache contents during garbage collection, using the recovery information to rebuild a forward lookup metadata structure that links a virtual address space to a physical address space for the source extent identified in the recovery information.

Images