Garbage Collection in Data Storage Systems

Abstract

To provide a method, a storage system and a program for overwriting data in place.SOLUTION: Various embodiments are directed to a method of garbage collection which is performed for a virtualized storage system whose virtual address space is addressed in extents. Valid data in source extents is copied via a cache into destination extents, and once all valid data in a source extent is copied into one or more destination extents, the source extent is reused. The source extent is released for reuse only after the one or more destination extents that received the valid data copied from the source extent are determined to be full, and the valid data copied from the source extent to the destination extent via the cache is flushed out of the cache.SELECTED DRAWING: Figure 3B

Description

[Technical Field]

[0001] The present invention relates to garbage collection in data storage systems. [Background technology]

[0002] A common strategy for performing writes in data storage systems is to always write to a new data range within the data domain, which has the advantage that it is possible to update a Redundant Array of Independent Disks (RAID) array without requiring a read from the disk.

[0003] A typical "write" would be performed as follows: An input-output (IO) is received for a given volume, with an address, size, and some data. The storage allocator allocates that volume to a storage location on the physical domain. The IO path then writes to that location on the physical domain. Once the write is complete, the forward and reverse lookup structures are updated and the IO is complete. As part of the forward lookup update, the journal is also updated.

[0004] A typical "read" would look at the forward lookup structure and identify where on the data domain the data is located. The data domain is then read and the data is returned to the user. Summary of the Invention

[0005] Various embodiments are directed to a method of garbage collection in a storage system in which a virtual address space is mapped to a physical address space, where the virtual address space is addressed in extents, and the storage system has one or more processors, multiple storage devices, and a cache memory. The method includes selecting a first destination extent as a current destination extent for receiving garbage collected data and selecting the first source extent as a current source extent to be garbage collected, where the current source extent contains one or more portions of valid data. The method also includes determining whether the destination extent selected as the current destination extent for receiving the garbage collected data is full. The method includes determining whether the source extent selected as the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to the destination extent. Additionally, the method includes, in response to determining that the current destination extent for receiving the garbage collected data is not full and determining that the current source extent to be garbage collected contains one or more portions of valid data that have not been copied to the destination extent, copying one of the portions of valid data that have not been copied to the destination extent from the current source extent to the current destination extent via the cache memory.Furthermore, the method includes, in response to determining that the current destination extent for receiving the garbage collected data is full, flushing the valid data to be copied to the current destination extent from the cache memory via the cache memory.Additionally, following the step of flushing valid data moved from the cache memory to the current destination extent, the method includes a step of releasing all valid data copied to the current destination extent via the cache memory so that it is available for writing to a physical address space mapped to the virtual address space of the one or more source extents.

[0006] Various embodiments of the garbage collection method may include, in response to determining that the current destination extent for receiving the garbage collected data is full, selecting a second destination extent as the current destination extent for receiving the garbage collected data.

[0007] Various embodiments of the garbage collection method further include, in response to determining that the second destination extent for receiving the garbage collected data is full, flushing valid data that was copied from the cache memory through the cache memory to the current destination extent, wherein the flushing of valid data that was copied from the cache memory through the cache memory to the current destination extent is flushed in the same order as the order in which the destination extent was determined to be full.

[0008] Various embodiments of the garbage collection method may further include, in response to determining that the current source extent to be garbage collected does not contain any portion of valid data that has not been copied to the destination extent, selecting a second source extent as the current source extent to be garbage collected.

[0009] Various embodiments of the garbage collection method include controlling the release of physical address space after flushing of the destination extent's cache using a sequence number assigned to each destination extent, each sequence number enabling lookup of a source extent for which garbage collection has been completed by copying data to that destination extent, the sequence number of the oldest destination extent for which flushing has not been performed being stored as a low watermark sequence number, the sequence number of the current destination extent allocated for garbage collection from the source extent being stored as a high watermark sequence number, the low watermark sequence number being incremented upon completion of flushing of the oldest destination extent from the cache memory, and the high watermark sequence number being incremented as further destination extents are selected for garbage collection.

[0010] Various embodiments of the garbage collection method include, in response to determining that the available virtual address space is less than a threshold, pausing the copying of valid data from the source extent to the destination extent, and releasing available for writing to physical address space mapped to the virtual address space of at least one destination extent having sequence numbers between values ​​of a low water mark sequence number and a high water mark sequence number.

[0011] Various embodiments of the garbage collection method may include maintaining a journal that stores recovery information for destination extents having sequence numbers between values ​​of a low watermark sequence number and a high watermark sequence number, the recovery information including a mapping between source extents and corresponding destination extents whose valid data was subject to garbage collection.

[0012] Various embodiments of the garbage collection method may further include, in response to determining a loss of cache contents during garbage collection, using the recovery information to rebuild a forward lookup metadata structure that links a virtual address space to a physical address space for a source extent identified in the recovery information.

[0013] Various embodiments are directed to a computer program product for garbage collection in a storage system in which a virtual address space is mapped to a physical address space, where the virtual address space is addressed in extents, and the storage system has one or more processors, multiple storage devices, and a cache memory.

[0014] Various embodiments relate to a storage system having a virtual address space mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the storage system including a storage allocator and a garbage collector.

[0015] Various embodiments can protect a data storage system against data loss related to a garbage collection that was in progress at the time of a cache failure. Various embodiments provide a garbage collection algorithm that is coordinated with a cache flush operation related to the garbage collected data. The garbage collection is performed such that old data (i.e., live data in a cache-flushed block) is not discarded until the storage allocator is confident that the live data being moved from the garbage collected block has been successfully flushed through the cache to its new physical memory location and has therefore safely arrived at and been stored in the destination block for the garbage collected live data.

[0016] Note that while various embodiments are designed to operate successfully in storage systems that use compression, embodiments may also be used in storage systems that do not use compression. Implementations of the present embodiments may be used for any data reduction that writes new data to a new location and relies on garbage collection. For example, uncompressed data may be written as proposed even when an overwrite to the same location would be successful.

[0017] The release of physical address space after flushing of the destination extent cache may be controlled in certain embodiments by assigning a sequence number to each destination extent, each sequence number enabling lookup of a source extent for which garbage collection has been completed by copying data to that destination extent, the sequence number of the oldest destination extent for which flushing has not been completed being stored as a low watermark sequence number, and the sequence number of the current destination extent allocated for garbage collection from the source extent being stored as a high watermark sequence number, where the low watermark sequence number is incremented upon completion of flushing of the oldest destination extent from the cache memory, and the high watermark sequence number is incremented as further destination extents are selected for garbage collection. If a shortage of available virtual address space is determined, this may be managed by pausing the copying of valid data from source extents to destination extents and releasing available for writes to the physical address space mapped to the virtual address space of at least one destination extent having a sequence number between the low water mark sequence number and the high water mark sequence number.

[0018] For recovery from accidental loss or corruption of the cache contents, it is advantageous to maintain a journal that stores recovery information for each sequence number between a low watermark sequence number and a high watermark sequence number, the recovery information including, for each sequence number, a mapping to a source extent whose garbage collection is completed by moving data to a destination extent associated with said sequence number. During garbage collection, if the cache contents are lost, the recovery information may be used to reconstruct forward lookup metadata structures that link virtual address spaces to physical address spaces for the source extent identified in the recovery information.

[0019] The method and storage system are particularly suitable, but not limited to, for use in conjunction with an operating system in which each write is performed to a new extent. The method and storage system are particularly suitable, but not limited to, for use with an operating system in which data is stored in compressed form.

[0020] In summary, various embodiments provide a garbage collection method for a virtualized storage system in which virtual address space is addressed by blocks referenced as extents. Old extents are garbage collected by transferring their still-valid data to new extents referenced as destination extents. The garbage-collected extents may then be freed for reuse by new writes. To account for potential cache losses during the process of garbage-collected data being written to physical memory, garbage-collected extents are initially withheld from being freed after these garbage collections. Once a destination extent receiving the garbage-collected data becomes full, a cache flush for that destination extent is initiated. The memory space of the garbage-collected extents associated with that destination extent is then freed, conditional on completion of the cache flush.

[0021] The invention will now be further described, by way of example only, with reference to exemplary embodiments illustrated in the drawings, in which: [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 1 illustrates a computer system environment with a mass storage device. [Figure 2] FIG. 1 is a block diagram showing the hardware structure of a data storage system. [Figure 3A] FIG. 1 is a flow diagram illustrating steps in a method according to the present disclosure. [Figure 3B] FIG. 1 is a flow diagram illustrating steps in a method according to the present disclosure. [Figure 4] 1A to 1D are schematic block diagrams illustrating the progress of garbage collection according to the method of the present disclosure. [Figure 5] FIG. 1 illustrates a cloud computing environment according to an embodiment of the present disclosure. [Figure 6] FIG. 2 illustrates an abstraction model layer according to an embodiment of the present disclosure. [Figure 7] 1 is a schematic diagram of a server connected to a storage system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0023] In the following detailed description, for purposes of explanation and not limitation, specific details are set forth in order to provide a better understanding of the present disclosure. It will be apparent to those skilled in the art that the present disclosure may be practiced in other embodiments that depart from these specific details.

[0024] As mentioned above, a common strategy for performing writes in data storage systems is to always write to a new data range within the data domain. This strategy has the advantage of allowing RAID (Redundant Array of Independent Disks) arrays to be updated without requiring reads from disk. However, always writing to a new data range results in a large amount of memory space being allocated, which must be reclaimed through garbage collection. In fact, if data compression is used, which is optional in some operating systems such as IBM® z / OS®, the size of the write will inevitably vary because its size depends on the compressibility of the data. For example, one write might be worth 8K of space, while another might only be 1K. Therefore, it is difficult to consistently overwrite data to the appropriate location when compression is used, because it cannot be guaranteed that the overwritten data will fit into the old memory location. Therefore, writes must always be performed to a new data range.

[0025] Garbage collection reclaims old data ranges that, as a result of overwrites, are dominated by obsolete data and contain only sparse live data. Garbage collection proceeds by salvaging the remaining live data and writing it to new data ranges in contiguous blocks. When data is overwritten, it is said to be invalidated. Garbage collection takes the region of the data domain with the invalidated data, moves the live data out of it, and writes it to a new location on the data domain, updating the forward lookup metadata in the process. The forward lookup metadata structure may be implemented as a tree, most commonly a b-tree. Forward b-tree structures map virtual memory addresses to physical memory addresses in virtual storage systems. The backward lookup metadata describes which volume references a particular range of the physical domain and therefore documents which data subranges are allocated and unallocated on the physical domain. The backward lookup metadata is then required by the garbage collection algorithm.

[0026] A journal is maintained that can be used to reconstruct the forward lookup metadata structure in the event of a cache failure. A cache failure typically results in the loss of many gigabytes of live data. Journaling ensures that all updates to the forward lookup metadata are recorded in a sequential log. This log may be replayed to reconstruct the forward lookup metadata. Integrating periodic copies of the forward lookup metadata (sometimes referred to as "snapshots") into this scheme eliminates the need for a potentially constantly growing journal size, since the journal only needs to track forward lookup updates since the last snapshot. If the forward lookup metadata has a tree structure, such as a b-tree, the periodic copying may be referred to as tree snapshotting. Note that the data referenced by the journal is typically not journaled itself due to the high write bandwidth cost of doing so. Approaches that journal the forward lookup metadata (e.g., b-tree updates) require much less bandwidth. The journaling and updating of the lookup structures may be performed by various modules or components in different embodiments. In various embodiments, the journaling and updating of the lookup structures may be performed by a storage allocator or by a garbage collector.

[0027] A typical "write" is performed as follows: An input-output (IO) is received for a given volume with an address, size, and some data. The storage allocator will allocate that volume to a storage location on the physical domain. The IO path then writes to that location on the physical domain. Once the write is complete, the forward and reverse lookup structures are updated and the IO is complete. The journal is also updated as part of the forward lookup update.

[0028] A typical "read" would look at the forward lookup structure and identify where on the data domain the data is located. The data domain is then read and the data is returned to the user.

[0029] In the event of a system failure, cache contents may be lost: If a cache exists below a component that implements data reduction techniques, in-flight data may be lost (data in flight is any data that has not yet been fully written to the media and would therefore potentially be lost if the cache were discarded). This is a common failure pattern in storage systems and can be tolerated or at least expected to some extent. This is the performance trade-off that occurs when using such a cache.

[0030] In storage systems with fixed physical domain locations for updates or thick volumes, cache loss may be tolerable because it only results in stale data being returned if the update was pending in cache at the time of the failure. However, in data-reduced volumes, it is difficult to write consistently to the same location during an overwrite, which means that data that was garbage collected at the time of the failure may also be lost, and this portion of the stale data may not be returned. In file systems, returning stale data is often preferable to returning a media error (data loss).

[0031] FIG. 1 illustrates an exemplary computer system 10. The computer system 10 includes a central processing unit (CPU) 12 connected to a mass storage device 14 and a memory device 16. The mass storage device may include a hard disk drive (HDD) device, which may be configured in a redundant array of independent disks (RAID). Garbage collection operations may be performed on device(s) 14 located on the system 10 or elsewhere. The memory device 16 may include an electrically erasable programmable read-only memory (EEPROM) or a host of related devices, such as RAM, ROM, and FLASH. The memory device 16 and mass storage device 14 are connected to the CPU 12 via a signal transmission medium. Additionally, the CPU 12 is connected via a communication port 18 to a communication network 20 having multiple additional attached computer systems 22 and 24. Computer system 10 may include one or more processor devices (e.g., CPU 12) and additional memory devices 16 for each individual component of computer system 10 to perform and implement garbage collection and other operations as described herein.

[0032] FIG. 2 is an exemplary block diagram 200 illustrating the hardware structure of a data storage system in a computer system for implementing an embodiment of the present invention. Host computers 210, 220, and 225 each function as a central processing unit for data processing as part of data storage system 200. Hosts (physical or virtual devices) 210, 220, and 225 may represent one or more new physical or logical devices for achieving the objectives of the present invention in data storage system 200. In one embodiment, by way of example only, data storage system 200 may be implemented as an IBM® System Storage® V9000®. Network connection 260 may be a Fibre Channel fabric, a Fibre Channel point-to-point link, a Fibre Channel over Ethernet fabric or point-to-point link, a FICON or ESCON I / O interface, any other I / O interface type, a wireless network, a wired network, a LAN, a WAN, heterogeneous or homogeneous, public (i.e., the Internet) or private, or any combination thereof. Hosts 210, 220, 225 may be local or distributed among one or more locations and may include any type of fabric (or fabric channel) or network adapter 260, such as Fibre Channel, FICON, ESCON, Ethernet, fiber optic, wireless, or coaxial adapter, to storage controller 240. Data storage system 200 therefore includes an appropriate fabric or network adapter 260 for communication. Data storage system 200 is illustrated in FIG. 2, including storage controller 240 and storage 230.

[0033] 2 as a single processing unit including a microprocessor 242, system memory 243, and non-volatile storage (NVS) 216. Storage controller 240 may, however, include multiple processing units, each with its own processor complex and system memory, interconnected by a dedicated network within data storage system 200. Storage 230 may be comprised of one or more storage devices, such as a storage array, connected to storage controller 240 by a storage network.

[0034] In some embodiments, the devices included in storage 230 may be connected in a loop architecture. Storage controller 240 manages storage 230 and facilitates processing of write and read requests intended for storage 230. System memory 243 of storage controller 240 stores operational software 250, program instructions, and data that processor 242 can access to perform functions and method steps associated with managing storage 230, including garbage collection. As shown in FIG. 2, system memory 243 includes internal memory cache 245, which is a cache for storage 230 (or alternatively, communicates with an external cache). The cache memory is used to buffer write and read data for write and read requests, respectively. In one embodiment, cache 245 is allocated to a device external to system memory 243 but remains accessible by microprocessor 242, which can function to provide additional security against data loss.

[0035] In some embodiments, the cache 245 is implemented with volatile and nonvolatile memory and is coupled to the microprocessor 242 via a local bus (not shown in FIG. 2 ) to improve performance of the data storage system 200. The NVS 216, included in the data storage controller 240, is accessible by the microprocessor 242 and provides additional support for operations, including garbage collection. The NVS 216 may be implemented with nonvolatile memory that does not require power to retain data and is therefore sometimes referred to as a persistent cache. The NVS 216 may be integrated into the cache 245. In some embodiments, a backup power source (not shown in FIG. 2 ), such as a battery, provides the NVS 216 with enough power to retain the data stored therein in the event of a power loss to the data storage system 200. In certain embodiments, the capacity of the NVS 216 is less than or equal to the total capacity of the cache 245.

[0036] Storage 230 may be physically composed of one or more storage devices, such as a storage array. A storage array is a logical grouping of individual storage devices, such as hard disks. In some embodiments, storage 230 is composed of a Just a Bunch of Disks (JBOD) array or a RAID array. A collection of physical storage arrays may be further combined to form ranks, which separate the physical storage from the logical configuration. Storage space within a rank may be allocated to logical volumes, which define the storage locations specified in write / read requests.

[0037] As shown in FIG. 2, a storage system may include logical volumes, or simply "volumes," and may have different types of allocations. Storage units 230a, 230b, and 230n are depicted as ranks in data storage system 200 and are referred to herein as ranks 230a, 230b, and 230n. Ranks may be local to data storage system 200 or may be located at physically separate locations. In other words, a local storage controller may connect with a remote storage controller and manage storage at a separate location. Rank 230a is shown configured with two full volumes 234 and 236 and one partial volume 232a. Rank 230b is shown with another partial volume 232b. Thus, volume 232 is allocated across ranks 230a and 230b. Rank 230n is shown as being assigned to volume 238 in its entirety, i.e., rank 230n references all physical storage for volume 238. From the above examples, it will be appreciated that a rank may be configured to include one or more partial volumes, or an entire volume, or both.

[0038] The storage controller 240 includes a garbage collection module 255 that assists in garbage collection in the journaled file system. The garbage collection module 255 operates in conjunction with the components of the storage controller 240, the hosts 210, 220, 225, and the storage device 230. The garbage collection module 255 may be located within the cache 245 or other components of the storage controller 240.

[0039] The storage controller 240 includes a control switch 241 that controls the Fibre Channel protocol to the host computers 210, 220, and 225, a microprocessor 242 that controls the entire storage controller 240, a non-volatile control memory 243 that stores a microprogram (operation software) 250 for controlling the operation of the storage controller 240, a cache 245 that temporarily stores (buffers) data, a buffer 244 that assists the cache 245 in reading and writing data, the control switch 241 that controls the protocol that controls data transfer to and from the storage device 230, and a garbage collection module 255 in which information can be set. Multiple buffers 244 may be implemented to assist garbage collection in a journaled file system.

[0040] The host computer or one or more physical or virtual devices 210, 220, and 255 and the storage controller 240 are connected via a network adapter 260 (which may be a Fibre Channel adapter) as an interface, i.e., via a switch referred to as a "fabric." A microprocessor 242 controls memory 243, which can store command information from the host device (physical or virtual) 210 and information identifying the host device (physical or virtual) 210. The control switch 241, buffer 244, cache 245, operation software 250, microprocessor 242, memory 243, NVS 216, and garbage collection module 255 communicate with each other. Some or all of the components, such as the operation software 245, may be embedded in memory 243 to perform garbage collection in a journaled file system. The components within the illustrated device may be coupled together and may communicate with each other.

[0041] FIG. 7 illustrates a server 710 connected to a storage system 712 in which the principles of the present invention may be implemented in various embodiments. The server 710 performs read and write operations to the storage system 712 via an interface, such as a standard interface such as SCSI (Small Computer System Interface). The storage system 712 has multiple connected physical components that provide processing and storage functionality. Primary long-term storage functionality is provided by a set of physical storage devices 714 (preferably consisting of a set of disks), which are divided into logically allocated regions 716. Additional memory is also provided within the storage system 712 by a write cache 718, which is used locally by the processing components of the storage system 712.

[0042] From the perspective of the external server 710, the storage system 712 presents one or more virtual disks 720 (vdisks) to the external server 710, which are a logical representation of the stored data. Data represented to the server 710 as residing on a single vdisk 720 may actually be stored on different disks 714 within the physical storage area 714. The mapping between vdisks 720 and physical disks 714 is managed by two components within the storage system 712: a forward lookup structure 722 and a reverse lookup structure 724. The forward lookup structure 722 maps one or more addresses within the vdisk 720 to the actual physical location(s) within the disks 714 where the data is stored. The reverse lookup structure 724 maps the physical location(s) within the disks 714 back to addresses within one or more vDisks 720.

[0043] When a read arrives at vdisk 720 from server 710, vdisk 720 communicates with forward lookup structure 722 to obtain the physical address of the data and reads the required data from the address in physical domain 714. For a read, no changes to either forward lookup structure 722 or reverse lookup structure 724 are required.

[0044] The behavior is more complicated when a write arrives from server 710 to vdisk 720. When new data is written to disk 714, forward lookup structure 722 must be updated to connect the address of the new data in vdisk 720 to the physical location of the data as it was actually stored in disk 714. In this case, both forward lookup structure 722 and reverse lookup structure 724 must be updated to reflect the connection between the data presented to server 710 by vdisk 720 and the actual physical location on disk 714 where the data is stored.

[0045] The storage system 712 also includes two additional components: a storage allocator 726 and a garbage collector 728. The storage allocator 726 performs several functions: first, it allocates live blocks from live regions 716 in the physical domain 714, which are communicated to the vdisk 720, and it maintains a free list for allocated data, described in more detail below. When new data is written to the storage system 712, the forward lookup structure 722 writes to a location determined by the allocated block location. The top-level storage allocator 726 has a set of "active" regions 716 that are tracked at any given time. A finite number of these regions 716 may be tracked. For example, in an exabyte physical domain 714 with a region size of (say) 1 gigabyte, this would result in tracking over 1 billion regions. Even if a region entry in the storage allocator 726 only cost 4B (i.e., it tracks how many blocks in a region 716 were available), this would consume 4 gigabytes of memory that would be better used as a cache. Thus, a set of active regions 716 is maintained up to a size of N regions 716, where N is the maximum number that can be handled by the cache 718. When all the storage in an active region 716 is used up, the entire region 716 leaves the set of active regions 716, and the memory used to track the regions 716 may be reclaimed and, if applicable, garbage collected by the garbage collector 728. When all N regions 716 are active, no more regions 716 can be allocated, which means that the free list (a list of unallocated blocks of memory) cannot receive additional host blocks. The garbage collector 728 essentially reclaims regions 716 with the limited live data that exists by rewriting the remaining live data to new regions 716, thereby releasing these reclaimed regions 716 to the storage allocator 726.A garbage collector 728 (or garbage collection module) connects to the backward lookup structure 724, which is used to perform garbage collection.

[0046] The storage system 712 is presented to the server 710 as thinly provisioned (also known as space-efficient), where the underlying storage can be much smaller than what is presented to the host (server 710). This is achieved through metadata structures that map virtual domain writes to the physical domain, and through compression and deduplication. Implementations that utilize compression must implement a mechanism for space recovery, known as garbage collection, because there is no guarantee that any two writes to the same virtual address will use the same amount of physical storage. Over time, the disk 714 is left with many small regions of free space, and it is desirable to coalesce these by moving writes in the physical domain (and updating their mapping in the virtual domain).

[0047] Next, the role of the journal is explained. The journal tracks updates into a b-tree, which stores mappings between virtual and physical addresses. The role of the journal is to track changes to these mappings if an error occurs during the update. The b-tree may then be restored by reference to the recovery information in the journal. Corruption of the mappings between the virtual and physical domains as a result of lost cache contents or other errors is avoided.

[0048] In the context of this overwrite operation, the steps involved in logging it to the journal are writing the new data to the new location, updating the b-tree to log the physical address of the new location, and marking the old physical data as invalid (stale). Garbage collection must then be performed to move the non-invalid data (i.e., data that is still valid) intermixed with the invalid data. Typically, the valid data in a particular virtual memory block will make up only a very small portion of the total, e.g., 10% or less. Garbage collection will move the still-valid data that may be the coldest (i.e., in main memory, on physical storage, no longer in cache memory), and the work of garbage collection will cause this cold data to become hot, i.e., in-flight in cache memory, and therefore at risk of loss in the event of a crash.

[0049] Various embodiments provide a garbage collection method for a virtualized storage system in which a virtual address space is mapped to a physical address space. The method may be implemented by modifying the system's storage allocator. The method can be used in storage systems that store data in a compressed format, such as IBM® z / OS®, as well as in uncompressed storage systems. The method is particularly intended for storage systems that constantly write to new virtual memory space to reduce reads, thereby consuming a lot of virtual memory space and requiring ongoing garbage collection. According to various embodiments, the storage allocator tracks the virtual memory used for writes in blocks, where each block is a range of virtual memory addresses. These blocks are called "extents."

[0050] In various embodiments, the garbage collection method proceeds as follows: The storage allocator selects an extent to receive the garbage collected data, which is referred to as the destination extent or target extent.

[0051] The storage allocator selects extents to be garbage collected, referred to as source extents, and the virtual addresses of the source extents are tagged as containing either valid or invalid data. As mentioned above, the amount of valid data is likely to be very small relative to the size of the source extents, i.e., much smaller than the amount of invalid data. In other words, each destination extent is likely to have enough space to receive all the valid data from several source extents, with 5-10 being a typical number.

[0052] The storage allocator, or in some embodiments, the garbage collector 728 (or garbage collection module 255), moves or copies valid data in the source extent to the destination extent via cache memory, then selects the second, third, and so on, for garbage collection, repeatedly, each time moving valid data to the destination extent via cache memory. This proceeds until the destination extent is full. The garbage collector 728 (or garbage collection module 255) may determine when the destination extent is full. When the destination extent is full, the remaining data that remains to be moved or copied from the source extent being garbage collected may be rolled, that is, copied to the next allocated destination extent. The entire source extent data from a given source extent need not fit into a single destination extent. This may be split across two, and in various embodiments, since flushing is done in order, it only needs to be tagged as dependent on the last one, thereby ensuring that both destination extents are flushed.

[0053] When the point is reached where the destination extent currently being used for garbage collection is full, the storage allocator or garbage collector selects another extent to be the next destination extent, thereby allowing for continued garbage collection. The process then repeats by filling the next destination extent with valid garbage collected data, and so on.

[0054] Asynchronously with the ongoing filling of successive destination extents and the emptying of any number of source extents of still-valid data, each full destination extent is flushed from the cache in turn. This flushing of each destination extent from the cache memory begins only as each such destination extent becomes full, in the order in which it was allocated and filled. Flushing and garbage collection transfers are controlled independently, so the amount of time it takes for each destination extent to be flushed may be slower or faster relative to the garbage collection rate. Because of the asynchronous nature of flushing and garbage collection transfers between source and destination extents, the virtual address space of the garbage-collected source extent is not immediately made available for new writes but is temporarily held. Freeing occurs first after each destination extent has been flushed, where "freeing" means making available for writes the physical address space mapped to the source extent's virtual address space whose valid data has been completely flushed from cache memory by the flushing of that particular destination extent. Thus, although the cache flushing and subsequent freeing of old addresses is not performed synchronously with the garbage collection from the source extent to the destination extent, these activities are coordinated.

[0055] Figure 3A shows the process flow of garbage collection activity. In step S1, a destination extent is selected to receive garbage collected data. In step S2, a source extent is selected for garbage collection. Steps S1 and S2 may be performed by a storage allocator. In step S3, valid data in the source extent is identified, and in step S4, this valid data is moved or copied to the destination extent. Steps S4 and S5 may be performed by a storage allocator or garbage collector. In step S5, if the destination extent is full (as determined by the storage allocator or garbage collector), the process flow returns to step S1 to select a new destination extent for continued garbage collection, and either the remaining data from the existing source extent or a new source extent is selected in step S2. The now-full destination extent is a candidate for cache flushing, as shown in Figure 3B.

[0056] Figure 3B shows the process flow for cache flushing and freeing garbage collected extents, i.e., source extents. In step F1, a full destination extent is selected for cache flushing. The garbage collector selects a full destination extent for cache flushing. In step F2, the garbage collector may initiate cache flushing. In step F3, once the cache flushing is complete as determined by the garbage collector in step F3, the storage allocator may free up the memory space mapped to the source extent that was just garbage collected into the destination extent that was just flushed for new writes in step F4.

[0057] More specifically, the method may be implemented such that every write to disk has a metadata header that documents which volume and forward-lookup logical block address (LBA) it originally belonged to. Once all metadata updates are complete, the backward-lookup structure, such as a b-tree, associated with a particular extent is updated and flushed to main memory (cache to disk) via cache memory. Once garbage collection is complete, the garbage-collected extent has its backward-lookup metadata updated to document that it has been garbage-collected; however, the garbage-collected extent is not immediately disposed of, i.e., made available for new writes immediately; rather, it is temporarily retained and remains available even if the contents of the cache are lost. The backward-lookup metadata for a garbage-collected extent is discarded only if the extent is subsequently reused or discarded.

[0058] Extents may be used by implementing multiple forward lookup structures, such as b-trees, at once. Completion of all metadata updates for an extent triggers a cache flush. Since writes are always written to a "new location" (either to newly allocated storage or to a previously garbage-collected extent), the cache effectively only acts as a temporary buffer for full-stride writes.

[0059] The storage allocator distinguishes between current extents being used to receive new writes inbound to the system and current extents being used internally to receive garbage collected data from other extents. The storage allocator tracks the set of extents that are currently active in garbage collection through the use of a pair of sequence numbers (in systems that support multiple storage nodes, a separate sequence number can be used for each node).

[0060] When garbage collection is performed on an extent, rather than immediately making the old extent available for reuse after it has been garbage collected, the old extent is temporarily retained, i.e., its availability as a new extent for writes is delayed. Because live data from a garbage-collected extent may be allocated to be moved to multiple destination extents, and tracking each of these is expensive, various embodiments track the extent currently being used for writes through a pair of sequence numbers, referred to herein as a low-watermark sequence number and a high-watermark sequence number. The use of low-watermark sequence numbers and high-watermark sequence numbers simplifies the control of freeing physical address space after a destination extent's cache is flushed. The storage controller assigns a unique sequence number to each destination extent. Each sequence number allows for lookup of a source extent that has completed garbage collection by moving data to that destination extent. The sequence number of the oldest destination extent that has not yet completed flushing is stored as the low-watermark sequence number. The sequence number of the destination extent currently allocated for garbage collection from the source extent is stored as the high water mark sequence number. The value of the low water mark sequence number is incremented when the oldest destination extent is flushed from cache memory, and the value of the high water mark sequence number is incremented when an additional destination extent is selected for garbage collection. Because the rate of flushing is variable and may be slower than the destination extents are filled over some periods and faster over other periods, the difference between the values ​​of the low water mark sequence number and the high water mark sequence number will increase and decrease over time.

[0061] Because destination extents are flushed in order of increasing sequence number, the low watermark sequence number always corresponds to the oldest destination extent that has not yet been flushed. The range of extents with indexes between the low watermark sequence number and the high watermark sequence number indicates those destination extents (and thus the garbage-collected extents that received garbage-collected data from them) that are at potential risk from cache content loss or other cache errors. It is safe to release garbage-collected extents that depend on a destination extent for new writes only if it is known with certainty that the destination extent has been flushed to disk.

[0062] The existence of low-water mark and high-water mark sequence numbers also simplifies what needs to be stored in the journal to protect against cache errors. The storage controller can limit the recovery information stored in the journal to memory addresses linked to those associated with sequence numbers between the low-water mark and high-water mark values. The recovery information for each such sequence number includes a mapping to the source extent whose garbage collection was completed by moving data to the destination extent associated with the sequence number. If the contents of the cache are lost during garbage collection, the recovery information is used to reconstruct the forward lookup metadata structure that links the virtual address space to the physical address space for the source extent identified in the recovery information. That is, the journal can attempt to recover the forward lookup structure through a replay approach. This replay approach reads the journal entry that indicates the physical address of the destination extent, then reads the destination extent in the physical domain and determines whether the location referenced by the journal entry has the correct metadata header belonging to the volume being recovered. If a discrepancy is found between the journal and data domains, journal replay continues with a later journal entry, and the address being recovered is left as a media error and subject to a later journal entry that finds more valid data. This replay approach attempts to find the last reference that has not been overwritten. Further details of this replay approach are disclosed in commonly owned, co-pending U.S. patent application Ser. No. 16 / 548,474, filed August 22, 2019, and entitled "Data Validity Verification During Data Recovery in a Log-Structured Array Storage System," which is incorporated by reference in its entirety. If a reference to a garbage collected extent is identified as recovered, the extent in question should be treated as if it was not successfully garbage collected and therefore requires a second garbage collection pass.Because the backward lookup metadata can mark restored references as invalid, extents must be marked for garbage collection without using the backward lookup structure. This is possible by utilizing the header data associated with each write. Garbage collection will inject all volume writes and allow the header to identify them to the IO path. The IO path will reject writes that arrive at locations that do not originally match the forward lookup structure. This is done so that it removes the requirement that the backward lookup structure exactly match the forward lookup structure for garbage collections with and without the backward lookup structure. Because it identifies and prevents the reuse of extents belonging to the garbage collected extent before flushing is complete, this reclamation approach will always find data belonging to the garbage collected extent in the event of a cache loss.

[0063] Because the proposed method retains garbage-collected extents longer than approaches that immediately reuse them, it will consume more memory space than conventional methods. Therefore, keeping in mind that a particular user, business, or application will typically allocate only a fixed amount of virtual memory space, using the proposed method may result in a shortage of virtual memory space. In particular, if flushing slows down for some reason, the difference between the low and high watermarks may become large, meaning that a large amount of memory space may be tied up in garbage collection. Consequently, if the amount of free virtual memory falls below a certain threshold—for example, below 10% or 5% of the total available virtual memory—it may be beneficial to stop garbage collection activity according to the proposed method, thereby allowing the virtual memory space allocated to garbage collection destination extents for those destination extents whose source extents have not yet been released to be freed and made available for new writes. If the storage allocator determines that there is insufficient available virtual address space, it suspends garbage collection by suspending the movement of valid data from source extents to destination extents. It can then free up the physical address space mapped to the virtual address space of destination extents with sequential numbers between the low and high water marks to make them available for writes. This freeing is not actually immediate, because it first notifies all parts of the storage allocator to stop using these destination extents. Once all storage associated with these destination extents has been reported as fully used or discarded, these destination extents can be freed up for new writes. Another option for dealing with the same problem is to cap the difference between the low and high water marks, imposing a maximum number of destination extents that can be associated with a garbage collection.If this cap is hit, garbage collection will be delayed until the low water mark sequence number is incremented, i.e., until the current cache flush is complete.

[0064] Figures 4(A) through 4(D) are schematic block diagrams illustrating an example of the progression of garbage collection using the proposed method. A timeline is shown from Figure 4(A) to Figure 4(D). Initially, virtual memory uses three example extents, Ext1, Ext2, and Ext3, with transfer to physical storage occurring via cache. Extent 1 (Ext1) contains a mix of cold data and stale (still valid) data to be garbage collected. Extent 2 (Ext2) contains static, cold data. Extent 3 (Ext3) is the extent currently being used for new writes. Extent 1 is then identified as a candidate for garbage collection based on the relatively small percentage of still valid data. The garbage collection process reads the extents and selects the data to retain (401). As shown diagrammatically in Figure 4(B), stale data from Extent 1 is identified for discarding, and the still-valid "live" data from Extent 1 is written to Extent 3. Figure 4(C) shows that even after garbage collection from Extent 1 to Extent 3 is complete, data is retained in Extent 1 while the garbage collected data potentially migrates through the cache to physical storage. Figure 4(D) shows how Extent 1 is discarded with respect to Extent 3 once the cache has been flushed to the physical storage device.

[0065] It will be apparent to those skilled in the art that all or part of the logical processing steps of the preferred embodiments may alternatively be embodied in a logical device or multiple logical devices that include logical elements configured to perform the logical processing steps of the method, and that such logical elements may include hardware components, firmware components, or a combination thereof.

[0066] It will also be apparent to those skilled in the art that all or part of the logic components of the preferred embodiments may alternatively be embodied in a logic device made up of logic elements to perform the steps of the present methods, and that such logic elements may include components such as logic gates in a programmable logic array or an application specific integrated circuit. Such logic arrangements are further embodied in enabling elements to temporarily or permanently establish logic structures in an array or circuit, for example using a virtual hardware description language, which may be stored or transmitted using a fixed or transmissible carrier medium.

[0067] In yet another embodiment, the present invention may be realized in the form of a computer-implemented method of deploying a service, comprising deploying a computer program operable to be deployed on a computer infrastructure and, when executed thereon, cause a computing device to perform all of the steps of the method.

[0068] It will be appreciated that the methods and components of the preferred embodiments may alternatively be embodied in whole or in part in a parallel computing system including two or more processors for executing parallel software.

[0069] Further embodiments of the present disclosure are computer program products defined with respect to systems and methods, which may include computer-readable storage medium(s) having computer-readable program instructions for causing a processor to implement aspects of the present invention.

[0070] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction-execution device.

[0071] The present invention may be a system, method, or computer program product, or combination thereof, at any level of technical detail possible in any integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0072] A computer-readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, punch cards, or mechanically encoded devices such as ridge structures in grooves with recorded instructions, and any suitable combination of the above. Computer-readable storage media, as used herein, is not to be construed as a transitory signal per se, such as an electric wave, a freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0073] The computer-readable program instructions described herein can be downloaded to each computing / processing device from a computer-readable storage medium or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within the respective computing / processing device.

[0074] Computer-readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source or object code written in any combination of one or more programming languages, including object-oriented languages ​​such as Smalltalk®, C++, or the like, and procedural languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions may execute as a stand-alone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, the electrical circuitry may execute the computer-readable program instructions by utilizing state information of the computer-readable program instructions to individualize the electrical circuitry, which may include, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), to perform aspects of the invention.

[0075] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer-readable program instructions that, when provided to a computer processor or other programmable data processing device, cause the instructions, executed by the computer processor or other programmable data processing device, to generate a machine-readable means for implementing the function(s) / acts identified in a block or blocks of the flowchart illustrations and / or block diagrams.

[0076] These computer-readable program instructions may also be stored on a computer-readable storage medium that can direct a computer, programmable data processing apparatus or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium having instructions stored thereon includes an article of manufacture containing instructions that implement an aspect of the function / act identified in a block or blocks of the flowchart illustrations and / or block diagrams.

[0077] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable data processing apparatus, or other device to perform a series of operational steps to generate a computer-implemented process such that the instructions executing on the computer, other programmable data processing apparatus, or other device implement aspects of the functions / actions identified in a block or blocks of the flowchart and / or blocks.

[0078] The flowchart diagrams and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart diagrams or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing particular logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order depicted in the figures. For example, two blocks shown in succession may actually be realized in a single step, executed simultaneously, substantially simultaneously, or with partial or full overlap, or the blocks may be executed in the reverse order depending on the functionality involved. It should be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs specific functions or actions or executes a combination of special-purpose hardware and computer instructions.

[0079] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings detailed herein is not limited to cloud computing environments. Rather, embodiments of the present invention may be implemented in conjunction with any other type of computing environment now known or hereafter developed.

[0080] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0081] The characteristics are as follows:

[0082] On-Demand Self-Service: Cloud consumers can unilaterally provision computing power, such as server time and network storage, as needed automatically without the need for human interaction with the service provider.

[0083] Broadband Network Access: Capabilities are available over the network and accessed via standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, PDAs).

[0084] Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with various physical and virtual resources dynamically allocated and reallocated according to demand. Consumers generally have no control over or knowledge of the exact location of the resources provided, but there is a sense of location independence in the sense that they can specify location at a higher level of abstraction (e.g., country, state, or data center).

[0085] Rapid Elasticity: Capacity can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the provisioned available capacity often appears externally unlimited, and any amount can be purchased at any time.

[0086] Metered Services: Cloud systems automatically control and optimize resource usage by utilizing metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active users). Resource usage is monitored, controlled, and reported, providing transparency to both providers and consumers of the services being used.

[0087] The service model is as follows:

[0088] Software as a Service (SaaS): The consumer is offered the ability to use a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the potential exception of limited user-specific application configuration settings.

[0089] Platform as a Service (PaaS): The ability offered to consumers is to deploy consumer-created or acquired applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the configuration of the application hosting environment.

[0090] Infrastructure as a Service (IaaS): The capability offered to the consumer is the provision of processing, storage, networking, and other basic computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating system, storage, deployed applications, and, in some cases, limited control over selected networking components (e.g., host firewalls).

[0091] The deployment model is as follows:

[0092] Private Cloud: Cloud infrastructure is used exclusively for one organization. It may be managed by the organization or a third party and may exist on-premise or off-premise.

[0093] Community Cloud: Cloud infrastructure is shared by several organizations to support a specific community with common concerns (e.g., mission, security requirements, policy and compliance considerations). It may be managed by the organization or a third party and may exist on-premises or off-premises.

[0094] Public Cloud: Cloud infrastructure is available to the general public or large industry organizations and is owned by organizations that sell cloud services.

[0095] Hybrid Cloud: A cloud infrastructure is a blend of two or more clouds (private, community, or public) that remain distinct entities but are joined by standardized or proprietary technologies that allow data and application portability (e.g., cloud bursting for load balancing between clouds).

[0096] Cloud computing environments are service-oriented with an emphasis on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0097] Referring now to FIG. 5, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10, with which local computing devices used by cloud consumers, such as, for example, a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof, may communicate. The nodes 10 may also communicate with each other. They may be physically or virtually grouped (not shown) in one or more networks, such as private, community, public, or hybrid clouds, as described above, or combinations thereof. This enables the cloud computing environment 50 to provide infrastructure, platform, or software, or combinations thereof, as a service, for which cloud consumers are not required to maintain resources on their local computing devices. The types of computing devices 54A-54N shown in FIG. 5 are for illustrative purposes only, and it will be understood that computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network, a network-addressable connection (e.g., using a web browser), or both.

[0098] An add-on according to an embodiment of the present disclosure may be installed in a web browser in the environment of Figure 5 as follows: One of the cloud computing nodes 10 may host a website from which an add-on may be downloaded, upon request, to a third-party computing device, such as any of computing devices 54A, 54B, and 54C. The request causes the add-on to be transmitted from node 10 to computing device 54A / 54B / 54C over a network connection, where the add-on is transmitted along with an installer for integration into a web browser already present on the computing device.

[0099] Referring now to Figure 6, a set of functional abstraction layers provided by cloud computing environment 50 (Figure 5) is shown. It will be understood in advance that the components, layers, and functions shown in Figure 6 are for illustrative purposes only, and embodiments of the present disclosure are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0100] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include a mainframe 61, a server based on a RISC (Reduced Instruction Set Computer) architecture 62, a server 63, a blade server 64, storage devices 65, and network and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0101] The virtualization layer 70 provides an abstraction layer from which examples of virtualized entities such as virtualized servers 71, virtualized storage 72, virtualized networks including virtual private networks 73, virtualized applications and operating systems 74, and virtual clients 75 are provided.

[0102] In one example, the management layer 80 may provide the following functions: Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides tracking of costs for resources utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides allocation and management of cloud computing resources to meet required service levels. Service level agreement (SLA) planning and fulfillment 85 provides pre-allocation and procurement of cloud computing resources in anticipation of future demands in accordance with SLAs.

[0103] The workload layer 90 provides examples of functionality for which a cloud computing environment is utilized. Examples of workloads and functionality provided from this layer include mapping and navigation 91, software lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and a storage allocator 96 with garbage collection management as described above in accordance with embodiments of the present disclosure.

[0104] It will be apparent to those skilled in the art that many modifications and variations can be made to the exemplary embodiments described above without departing from the scope of the present disclosure.

Claims

1. 1. A method of garbage collection in a storage system in which a virtual address space is mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the method comprising: selecting a current destination extent for receiving the garbage collected data; selecting a current source extent containing one or more portions of valid data to be garbage collected; determining whether the current destination extent is full; determining whether the current source extent contains one or more portions of valid data that have not been copied to a destination extent; in response to determining that the current destination extent is not full and determining that the current source extent contains one or more portions of valid data that have not been copied to the destination extent, copying one of the portions of valid data that have not been copied to the destination extent from the current source extent to the current destination extent via the cache memory; asynchronously with the copying step, initiating a cache flush for the current destination extent in response to determining that the current destination extent is full, thereby flushing the valid data copied from the cache memory to the current destination extent via the cache memory to a corresponding storage device among the plurality of storage devices; a step of releasing, asynchronously with the copying step, the physical address space mapped to the virtual address space of one or more source extents from which valid data has been copied to the current destination extent via the cache memory, so that the physical address space is available for writing, following the step of flushing the valid data copied from the cache memory to the current destination extent; A method comprising:

2. 2. The method of claim 1, wherein the releasing step is performed after, for one source extent among the one or more source extents, if there are multiple destination extents that have received valid data from the one source extent, the multiple destination extents have been flushed.

3. 3. The method of claim 1, wherein the freeing step is controlled based on tracking extents that are currently active in the garbage collection.

4. 4. The method of claim 1, further comprising the step of controlling the release of the physical address space after flushing of the destination extent's cache by assigning a sequence number to each destination extent, each sequence number enabling a lookup of the source extent whose garbage collection has been completed by copying data to that destination extent.

5. the sequence number of the oldest destination extent for which flushing has not occurred is stored as a low water mark sequence number; The sequence number of the current destination extent allocated for garbage collection from the source extent is stored as a high water mark sequence number; the low water mark sequence number is incremented upon completion of flushing of the oldest destination extent from the cache memory; the high water mark sequence number is incremented as more destination extents are selected for garbage collection; The method of claim 4, comprising:

6. A computer program for garbage collection in a storage system in which a virtual address space is mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the computer program comprising: selecting a current destination extent for receiving the garbage collected data; selecting a current source extent containing one or more portions of valid data to be garbage collected; determining whether the current destination extent is full; determining whether the current source extent contains one or more portions of valid data that have not been copied to a destination extent; in response to determining that the current destination extent is not full and determining that the current source extent includes one or more portions of valid data that have not been copied to a destination extent, copying one of the portions of valid data that have not been copied to a destination extent from the current source extent to the current destination extent via the cache memory; asynchronously to said copying, initiating a cache flush for said current destination extent in response to determining that said current destination extent is full, thereby flushing said valid data copied from said cache memory to said current destination extent via said cache memory to a corresponding storage device among said plurality of storage devices; subsequent to the flushing of the valid data copied from the cache memory to the current destination extent, asynchronously with the copying, releasing the physical address space mapped to the virtual address space of one or more source extents from which valid data has been copied via the cache memory to the current destination extent so that the physical address space is available for writing; A computer program that executes the following:

7. A storage system having a virtual address space that is mapped to a physical address space, the virtual address space being addressed in extents, the storage system having one or more processors, a plurality of storage devices, and a cache memory, the storage system comprising: a storage allocator for selecting a current destination extent for receiving the garbage collected data; and the storage allocator for selecting a current source extent containing one or more portions of valid data to be garbage collected; a garbage collector that determines whether the current destination extent is full; the garbage collector determining whether the current source extent contains one or more portions of valid data that have not been copied to a destination extent; the garbage collector for copying, in response to determining that the current destination extent is not full and determining that the current source extent contains one or more portions of valid data that have not been copied to the destination extent, one of the portions of valid data that have not been copied to the destination extent from the current source extent to the current destination extent via the cache memory; asynchronously with the copying, the garbage collector for initiating a cache flush for the current destination extent in response to determining that the current destination extent is full, thereby flushing the valid data copied from the cache memory to the current destination extent via the cache memory to a corresponding storage device among the plurality of storage devices; the storage allocator for, following flushing the valid data copied from the cache memory to the current destination extent, asynchronously with the copying, releasing the physical address space mapped to the virtual address space of one or more source extents for which valid data has been copied via the cache memory to the current destination extent so as to be available for writing; a storage system,

Images