Location-based caching on magnetic tape

Abstract

A system and method for caching data on a magnetic storage tape is described, comprising a plurality of deduplication tapes, a plurality of data objects stored on the plurality of deduplication tapes, each data object comprising at least two segments, each segment is stored at at least two different positions on at least one of the tapes, an index stored on a fast storage medium indicating which data object of the plurality of data objects the segment is used and the position of the segment on the tape, a processor, and a processor. And a cache on the fast storage medium for storing segments based on the cost, based at least on the index: identifying a cluster of segments located close to each other on at least one tape, the segments in the cluster belonging to at least one data object, and calculating a cost for recovering each stored segment. Related methods and systems are also described.

Description

[0001] Technical Field and Background

[0002] In some embodiments of the present invention, the present invention relates to a method and system for caching data stored on a magnetic storage tape, and more particularly, but not exclusively, to a method and system for determining which data stored on a magnetic storage tape to move to a cache.

[0003] The demand for data storage is growing, exceeding the capacity of magnetic disks. Due to the lower cost and longer retention of magnetic tape, the focus of secondary storage and archiving is shifting from deduplication-enabled disk to magnetic tape technology. Due to the relatively long seek times (tens of seconds) of magnetic tape, most storage applications use magnetic tape without any optimization other than built-in compression. The introduction of deduplication in magnetic tape makes it even more affordable, but increases seek times when reading data.

[0004] Deduplication systems and methods for data stored on disks (e.g., solid state drives (SSDs), hard disk drives (HDDs), etc.) are widely used to ensure that excess copies of duplicate data are eliminated. Doing so can significantly reduce storage capacity requirements. However, such disk-based storage methods and systems are not easily adaptable to magnetic storage tapes. Typically, when data is backed up to magnetic storage tapes, the data is backed up in its original form and no deduplication is performed. Some applications limit the deduplication domain to a silo, which contains all the deduplicated data. However, this technique has a disadvantage in that all the deduplicated data is restored upon read, even if only a portion of the data is needed. Summary of the Invention

[0005] The present invention is directed to improving performance when recovering data stored on magnetic storage tapes.

[0006] According to an aspect of some embodiments of the present invention, there are provided a plurality of magnetic storage tapes, the plurality of magnetic storage tapes having been deduplicated, a plurality of data objects stored on the plurality of magnetic storage tapes, each of the plurality of data objects comprising at least two segments, each of the at least two segments being stored at at least two different locations on at least one magnetic storage tape among the plurality of magnetic storage tapes, an index, wherein for each segment of the plurality of data objects, the index indicates in which data object among the plurality of data objects the segment is used and the location of the segment on the plurality of magnetic storage tapes, wherein the index is stored on a fast storage medium, a processor for, based at least on the index: identifying clusters of segments located close to each other on at least one magnetic storage tape among the plurality of magnetic storage tapes, wherein the segments in the clusters belong to at least one data object among the plurality of data objects, and calculating a cost for recovering each stored segment stored on the plurality of magnetic storage tapes, each segment being part of at least one data object, and a cache for storing the segments based on the costs calculated by the processor, wherein the cache is stored on the fast storage medium.

[0007] According to some embodiments of the present invention, when restoring a data object to which the segment belongs, the cost is based at least in part on an amount of time required to perform a seek to an associated storage segment of the segment.

[0008] According to some embodiments of the present invention, each data object includes in its metadata a user-defined maximum acceptable time for restoring the data object upon request.

[0009] According to some embodiments of the present invention, each segment includes in its metadata a weight based on the number of data objects that use the segment.

[0010] According to some embodiments of the present invention, if the weight exceeds a given threshold, the data object is loaded into the cache.

[0011] According to some embodiments of the present invention, each segment includes a fingerprint in its metadata, the fingerprint comprising a strong hash value.

[0012] According to some embodiments of the present invention, the fast storage medium includes a plurality of solid-state hard drives.

[0013] According to some embodiments of the present invention, the plurality of magnetic storage tapes are included in one or more deduplication domains.

[0014] According to some embodiments of the present invention, the at least two segments are stored on more than one magnetic storage tape.

[0015] According to one aspect of some embodiments of the present invention, there is provided a method for deduplicating data on a plurality of magnetic storage tapes, wherein a plurality of data objects are stored on the plurality of magnetic storage tapes, each of the plurality of data objects including at least two segments, each of the at least two segments being stored at at least two different locations on at least one magnetic storage tape among the plurality of magnetic storage tapes, an index being performed on each segment of the plurality of data objects, the index at least indicating in which data object among the plurality of data objects the segment is used and the location of the segment on the plurality of magnetic storage tapes, wherein the index is stored on a fast storage medium, a processor identifying, based at least on the index, clusters of segments located close to each other on at least one magnetic storage tape among the plurality of magnetic storage tapes, the segments in the cluster belonging to at least one data object among the plurality of data objects, the processor identifying segments used by more than one data object, calculating a cost of recovering each stored segment stored on the plurality of magnetic storage tapes, each segment being part of at least one data object, and storing the segments in a cache based on the cost, wherein the cache is stored on the fast storage medium.

[0016] According to some embodiments of the present invention, when restoring a data object to which the segment belongs, the cost is based at least in part on an amount of time required to perform a seek to an associated storage segment of the segment.

[0017] According to some embodiments of the present invention, each data object includes in its metadata a user-defined maximum acceptable time for restoring the data object upon request.

[0018] According to some embodiments of the present invention, each segment includes in its metadata a weight based on the number of data objects that use the segment.

[0019] According to some embodiments of the present invention, if the weight exceeds a given threshold, the data object is loaded into the cache.

[0020] According to some embodiments of the invention, each segment is fingerprinted in its metadata, the fingerprint comprising a strong hash value.

[0021] According to some embodiments of the present invention, the fast storage medium includes a plurality of solid-state hard drives.

[0022] According to some embodiments of the present invention, the plurality of magnetic storage tapes are included in one or more deduplication domains.

[0023] According to some embodiments of the present invention, the at least two segments are stored on more than one magnetic storage tape.

[0024] According to an aspect of some embodiments of the present invention, there is provided a computer program for caching data objects stored on magnetic storage tapes, the computer program comprising program instructions that, when executed by at least one processor, cause the at least one processor to: deduplicate data on a plurality of magnetic storage tapes, store a plurality of data objects on the plurality of magnetic storage tapes, each of the plurality of data objects comprising at least two segments, each of the at least two segments being stored at at least two different locations on the plurality of magnetic storage tapes, index each segment of the plurality of data objects, the index indicating at least in which of the plurality of data objects the segment is used and the location of the segment on the plurality of magnetic storage tapes, wherein the index is stored on a fast storage medium, the processor identifying, based at least on the index, a cluster of segments located close to each other on at least one of the plurality of magnetic storage tapes, the segments in the cluster belonging to at least one of the plurality of data objects, the processor identifying segments used by more than one data object, calculate a cost of recovering each stored segment stored on the plurality of magnetic storage tapes, each segment being part of at least one data object, and store the segments in a cache based on the determination by the processor, wherein the cache is stored on the fast storage medium.

[0025] According to one aspect of some embodiments of the present invention, there are provided: a plurality of magnetic storage tapes, a plurality of data objects stored on the plurality of magnetic storage tapes, each of the plurality of data objects including at least two segments, each of the at least two segments being stored in at least two different locations on the plurality of magnetic storage tapes, a processor for: calculating a cost of recovering each stored data segment stored on the plurality of magnetic storage tapes, each data segment being a part of at least one data object, sorting the data segments from highest recovery cost to lowest recovery cost based on the calculation, loading the data segment with the highest cost from one of the plurality of magnetic storage tapes into a cache, wherein, if a remaining available size in the cache is greater than a size of the data segment with the highest cost, the cache is located on a fast storage medium, and subtracting the size of the data object segment loaded into the fast storage medium from the remaining available size in the cache.

[0026] According to some embodiments of the present invention, the cost is calculated based on at least the following: commonality of each stored data segment of multiple data objects including the at least one data object, time required to recover the stored data segment from a magnetic storage tape storing the stored data segment, and a service level agreement including a user-defined maximum acceptable time for recovering the data object upon request.

[0027] According to some embodiments of the invention, the cost is weighted to take into account the number of data objects using the segment.

[0028] According to some embodiments of the present invention, each segment includes a fingerprint in its metadata, the fingerprint comprising a strong hash value.

[0029] According to some embodiments of the present invention, the fast storage medium includes a plurality of solid-state hard drives.

[0030] According to some embodiments of the present invention, the plurality of magnetic storage tapes have been deduplicated in one or more deduplication domains.

[0031] According to one aspect of some embodiments of the present invention, a method is provided for: calculating a cost of recovering each stored data segment stored on a plurality of magnetic storage tapes, each data segment being part of at least one data object, sorting the data segments from highest recovery cost to lowest recovery cost based on the calculation, loading the data segment with the highest cost from one of the plurality of magnetic storage tapes into a cache, wherein, if a remaining available size in the cache is greater than a size of the data segment with the highest cost, the cache is located on a fast storage medium, and subtracting the size of the data object segment loaded into the fast storage medium from the remaining available size in the cache.

[0032] According to some embodiments of the present invention, the cost is calculated based on at least the following: commonality of each stored data segment of multiple data objects including the at least one data object, time required to recover the stored data segment from a magnetic storage tape storing the stored data segment, and a service level agreement including a user-defined maximum acceptable time for recovering the data object upon request.

[0033] According to some embodiments of the invention, the cost is weighted to take into account the number of data objects using the segment.

[0034] According to some embodiments of the invention, each segment is fingerprinted in its metadata, the fingerprint comprising a strong hash value.

[0035] According to some embodiments of the present invention, the fast storage medium includes a plurality of solid-state hard drives.

[0036] According to one aspect of some embodiments of the present invention, a computer program for caching data objects stored on magnetic storage tapes is provided, the computer program comprising program instructions which, when executed by at least one processor, cause the at least one processor to: calculate a cost of recovering each stored data segment stored on a plurality of magnetic storage tapes, each data segment being part of at least one data object, sort the data segments based on the calculation from highest recovery cost to lowest recovery cost, load the data segment with the highest cost from one of the plurality of magnetic storage tapes into a cache, wherein, if a remaining available size in the cache is greater than a size of the data segment with the highest cost, the cache is located on a fast storage medium, and the size of the data object segment loaded into the fast storage medium is subtracted from the remaining available size in the cache.

[0037] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as those known to those of ordinary skill in the art to which the present invention belongs. Although methods and materials similar or identical to those described herein can be used in the practice or testing of embodiments of the present invention, only exemplary methods and / or materials are described below. In the event of a conflict, this patent specification (including definitions) shall prevail. In addition, these materials, methods, and examples are merely illustrative and are not necessarily restrictive.

[0038] The implementation of the method and / or system provided by the embodiments of the present invention may involve performing or completing the selected tasks manually, automatically, or in combination with the two. In addition, the actual instruments and equipment according to the method and / or system embodiments of the present invention may utilize hardware, software, firmware, or a combination of the three to complete multiple selected tasks through an operating system.

[0039] For example, the hardware for performing selected tasks according to embodiments of the present invention may be implemented as a chip or circuit. For software, the execution of selected tasks according to embodiments of the present invention may be performed by a computer executing multiple software instructions through any suitable operating system. In an exemplary embodiment of the present invention, one or more tasks in the exemplary embodiments of the method and / or system described herein are performed by a data processor, for example, a computing platform for executing multiple instructions. Optionally, the data processor includes volatile memory and / or non-volatile memory for storing instructions and / or data, for example, a hard disk and / or removable media for storing instructions and / or data. Optionally, a network connection is also provided. Optionally, a display and / or user input device such as a keyboard or mouse is also provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Some embodiments of the present invention will be described herein by way of example only, with reference to the accompanying drawings. With specific reference to the accompanying drawings, it should be emphasized that the details shown are by way of example only and are provided for illustrative purposes only. Thus, based on the accompanying drawings, it will be apparent to those skilled in the art how to implement the embodiments of the present invention.

[0041] In the attached figure:

[0042] Figure 1A and Figure 1B A system for location-based caching of data stored on a magnetic storage tape is shown;

[0043] Figure 2 is a block diagram of an exemplary architecture of a computer 200 for caching data stored in Figure 1A and Figure 1B A system for storing data on magnetic tape;

[0044] Figure 3 It is used to store cache Figure 1A and Figure 1B A flow chart of a first method of storing data on a magnetic tape;

[0045] Figure 4 It is used to store cache Figure 1A and Figure 1B A flow chart of a second method of storing data on a magnetic tape. DETAILED DESCRIPTION

[0046] In some embodiments, the present invention relates to a method and system for caching data stored on a magnetic storage tape, and more particularly, but not exclusively, to a method and system for determining which data stored on a magnetic storage tape to move into a cache.

[0047] Before explaining at least one embodiment of the present invention in detail, it should be understood that the present invention is not necessarily limited in its application to the details of the structure and arrangement of the components and / or methods set forth in the following description and / or illustrated in the drawings and / or examples. The present invention may have other embodiments or may be practiced or carried out in various ways.

[0048] Referring now to the accompanying drawings, Figure 1A and Figure 1BA system for location-based caching of data stored on magnetic storage tapes is shown. A plurality of data objects are shown, here represented as OBJ01 101, OBJ02 102, ..., OBJ-n 103. Also shown are a plurality of magnetic storage tapes MST00 110, MST01 111, MST02 112, MST03 113, and MST-n 119. The plurality of data objects 101, 102, 103 are stored as segments of the respective data objects on the plurality of magnetic storage tapes 110, 111, 112. Data segments 121, 122, 123 are shown as comprising data object OBJ01 101. Data segments 124, 125, 126 are shown as comprising data object OBJ02 102. Data segments 127, 128, 129 are shown as comprising data object OBJ-n 103.

[0049] As an example, data segment 121 and data segment 129 are shown as being stored on MST00 110. Data segment 123 and data segment 125 are shown as being stored on MST01 111. Data segment 126 is shown as being stored on MST02 112. Data segment 124 and data segment 129 are shown as being stored on MST03 113. Data segment 122 and data segment 128 are shown as being stored on MST-n 119. It will be appreciated that additional copies of the various segments may be stored on other magnetic storage tapes. Figure 1A The description is intended only as an illustrative example and is not intended to be limiting in any way. Furthermore, since not all magnetic storage tapes and all data objects are shown in the figure but omitted, as indicated by "..." in the figure, it is understood that data objects not shown in this example may be copied to magnetic storage tapes not shown. In practice, each segment may be stored in at least two locations on each of the magnetic storage tapes 110, 111, 112, 113, ..., 119.

[0050] The plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119 have been deduplicated at some point in time, so that ideally, a small number of copies or only one copy of each data segment is stored on the magnetic storage tapes. Figure 1A As can be seen from the description, each of the plurality of data objects 121, 122, ..., 129 includes at least two segments. For ease of description, each data object in this example is shown as having three data segments only as an example and in a non-limiting manner.

[0051] Now refer to Figure 1BAn index 140 is stored on a fast storage element 130 (which may include one or more of a solid state drive (SSD), a hard disk drive (HDD), a flash drive, etc., or a combination thereof). Index 140 includes at least one entry for each segment, for example, segments 121, 122, ..., 129 of multiple data objects 101, 102, 103. The entry indicates in which of the multiple data objects the segment is used and the location of the segment on multiple magnetic storage tapes. As an example, OBJ01 101 has three entries in the index, one for each of the three segments 121, 122, and 123 of OBJ01 101. These entries list the magnetic storage tape on which each of these segments is stored. As an example, segment 121 of OBJ01 is shown as being stored on magnetic storage tape MST00110. The index also provides the location of the segment on the tape, as explained below. Continuing with the example of segment 121 of OBJ01 on MST00110, index 140 shows the location of segment 121 as being on band 1, wrap 11, at offset 3172 on MST00. All data tracks in a wrap are written and read together, so for positioning purposes, the tracks themselves are not a useful way to find a position. Accordingly, the position in a given wrap is provided as a longitudinal position (LPOS), that is, as an offset along the wrap. It will be understood that the "offset along the wrap" does not represent bytes, i.e., an amount of data, but rather the physical length of the wrap along the tape of the magnetic storage tape. It will also be understood that the numbers provided in the offset column of index 140 are for example and illustration only and are not intended to limit the scope of the claims herein.

[0052] Return Reference Figure 1A , the magnetic storage tape is arranged with four data bands 141 sandwiched between five narrow servo bands 145. The tape head has 8, 16, or 32 data read / write head elements and two servo read elements. A collection of 8, 16, or 32 tracks is read or written in a single, unidirectional, end-to-end stroke, called a "reel." The tape head moves laterally to access a different reel within each tape, as well as to access other tapes. For exemplary purposes only, reel 150 of tape 1 141 is shown. For exemplary purposes only, tracks 160 of reel 12 of tape 1 are also shown.

[0053] return Figure 1B, the processor is configured to identify clusters of segments located proximate to each other on at least one magnetic storage tape of the plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119. Based on identifying these clusters, the processor 150 may then calculate a cost 160 for recovering each stored segment 121, 122, ..., 129 stored on the plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119. The cost 160 may be calculated based on an amount of time required to perform a seek to the identified segment 121, 122, ..., 129 on the corresponding magnetic storage tape.

[0054] For example, segments 125 and 126 are both segments of the same object OBJ02. By examining index 140, it can be determined that segment 125 is located at tape 2, reel 24, LPOS 1637 on magnetic storage tape MST02, and segment 126 is located at tape 2, reel 50, LPOS 0991 on magnetic storage tape MST02. Therefore, when retrieving these two segments 125 and 126, the retrieval time for each segment is not significantly increased because they are stored clustered together. In contrast, segments 127 and 128 are both part of object OBJ-n. By examining index 140, it can be determined that segment 127 is located at tape 3, reel 23, LPOS 1751 on magnetic storage tape MST00, and segment 128 is located at tape 1, reel 34, LPOS 98880 on magnetic storage tape MST-n. Therefore, when retrieving these two segments 127, 128, the increase in retrieval time for each of the two segments 127, 128 is much greater than the increase in retrieval time for retrieving two aggregated stored segments (which will not increase significantly) because the two segments 127, 128 are not even stored on the same magnetic storage tape.

[0055] Therefore, the processor 150 may move one or both of the two segments with higher retrieval costs to a cache stored on a fast storage device such as an SSD or HDD.

[0056] In some embodiments, each of the plurality of data objects 101, 102, 103 may include in its metadata a user-defined maximum acceptable time for restoring a data object in the plurality of data objects 101, 102, 103 upon request. Because partial segments 121, 122, ..., 129 may be used in more than one object, each segment 121, 122, ..., 129 may include in its metadata a weight based on the number of data objects that use the segment. For example, if segment 129 is used in three data objects (only one of which is shown), segment 129 may include these three data objects in its associated metadata. Therefore, the processor may determine that segment 129 is likely to be restored more frequently than other segments 121, 122, ..., 128 that are used in fewer than three data objects. Therefore, the processor may determine that the restore cost of segment 129 is higher and, therefore, determine that segment 129 should be copied to cache 170. For example, there may be a given threshold (relative to the cost of other segments) such that if the determined cost 160 is above the given threshold, the processor copies the segment to cache 170.

[0057] Segments 121, 122, 123, ..., 129 may have fingerprints in their associated metadata, as will now be explained. When executed on segments 121, 122, 123, ..., 129, the fingerprint for each segment may comprise the output of a hash function. Hash functions include SHA1, SHA256, MD5, Tiger, and Blake3, among others. An index for a portion of each fingerprint may be created from the complete index of the fingerprint. The resulting magnetic storage tape may then be loaded to read the data into the cache.

[0058] Various magnetic storage tapes may include one or more deduplication domains, so data found on one magnetic storage tape may not have been deduplicated with respect to data found on a second magnetic storage tape. This may result in a segment or its versions existing on more than one magnetic storage tape.

[0059] In another embodiment, the processor may sort the segments 121, 122, 123, ..., 129 from highest restore cost to lowest restore cost based on the calculated cost 160. If the remaining available size in the cache is greater than the size of the data segment with the highest restore cost, the segment with the highest restore cost may be copied to the cache 170. The processor 150 may then deduct the size of the data object loaded to the fast storage medium from the remaining available size in the cache from its associated memory. If the processor 150 determines that there is enough space remaining in the cache to store the next segment with the highest restore cost in the cache 170, the above process may be repeated.

[0060] In this case, the cost 160 is calculated based on at least the following: the commonality of each of the stored segments 121, 122, 123, ……, 129 of the multiple data objects 101, 102, 103; the time required to recover the segment from the magnetic storage tapes 110, 111, 112, 113, ……, 119 storing the segment when recovering the data object to which the segment belongs; the service level agreement including the user-defined acceptable maximum time for recovering the data objects 101, 102, 103 according to a request.

[0061] As an example:

[0062] Let K be the size of the cache;

[0063] Let X be a data segment / portion of size Y, where Y < K;

[0064] Let X be involved in the data objects A1 to An;

[0065] Let S1(X) to SN(X) be the recovery times of A1 to An respectively in the case of needing to seek to X;

[0066] Let NS1(X) to NSN(X) be the recovery times of A1 to An respectively in the case of not needing to seek to X;

[0067] Let SLA1 to SLAn be the SLAs of the data objects A1 to An respectively;

[0068] Let W(X) represent the degree of "commonality" of X (simply defined as W(X) = the number of data objects in which X is involved divided by a certain factor, that is, W(X) = n / C, to avoid the commonality weight having too much influence in the calculation).

[0069] Then:

[0070] 1.

[0071] 2. Repeat step 1 for all data segments of size < K.

[0072] 3. Sort the data segments in descending order of cost.

[0073] 4. Starting from the data segment with the highest cost:

[0074] a. If the remaining available size in the cache is greater than the size of the current data segment / portion, load the data segment into the cache.

[0075] b. Update the remaining available size in the cache (subtract the loaded size of the data segment from it).

[0076] Now refer to Figure 2 , Figure 2 is a block diagram of an exemplary architecture of a computer 200 for caching data stored in Figure 1A and Figure 1B A system for storing data on magnetic tape in a magnetic storage system. Computer 200 is suitable for implementing any of the above-described systems, methods, or processes. Computer 200 may include one or more processors, such as processor 201. Processor 201 may be the same as or similar to processor 150 described above. Processor 201 provides an execution platform for executing machine-readable instructions, such as software. One of processors 201 may be a dedicated processor for performing the caching operations on the tape system described above.

[0077] Commands and data from the processor 201 can be communicated via a communication bus 202. The computer 200 may include a main memory 203 (e.g., random access memory (RAM) 204, where machine-readable instructions may reside during operation) and a secondary memory 205. The secondary memory 205 may include, for example, a hard disk drive 206 and / or a solid-state drive 207, and / or a removable storage drive 208 (which is typically not regularly accessed but may be accessed by service personnel or installers, etc.), such as a floppy disk drive, a tape drive 210, an optical drive, a flash drive, etc., or a non-volatile memory that can store a copy of the machine-readable instructions or software. The secondary memory 205 may also include a read-only memory (ROM), an erasable programmable ROM (EPROM), or an electrically erasable programmable ROM (EEPROM). In addition to software, data related to caching of data stored on magnetic storage tapes or other similar data may be stored in main memory 203 or secondary memory 205. Removable storage drive 208 may read from or write to removable storage unit 209.

[0078] A user can interact with the computer 200 through a user interface, which includes an input device 211, such as a touch screen, keyboard, mouse, stylus, etc., as well as an interface for input via a wireless interface, to provide user input data or other commands. A display adapter 215 interacts with the communication bus 202 and the display 217, receives display data from the processor 201, and converts the display data into display commands for the display 217.

[0079] The network interface 219 is used to communicate with other systems and devices via a network (e.g., a wireless network). The network interface 219 typically includes a wireless interface for communicating with wireless devices in the wireless community. A wired network interface (e.g., an Ethernet interface) may also be present. The computer 200 may also include other interfaces, including but not limited to Bluetooth or a High-Definition Multimedia Interface (HDMI).

[0080] Now refer to Figure 3 , Figure 3 It is used to store cache Figure 1A and Figure 1B Flowchart of a first method for deduplicating data on magnetic storage tapes in a plurality of magnetic storage tapes. In step 310, deduplication is performed on a plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119. In step 320, a plurality of data objects are stored on the plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119, wherein each data object 101, 102, 103 in the plurality of data objects includes at least two segments 121, 122, 123, ..., 129, and each of the at least two segments 121, 122, 123, ..., 129 is stored in at least two different locations on at least one magnetic storage tape in the plurality of magnetic storage tapes. In step 330, each segment 121, 122, 123, ..., 129 of multiple data objects 101, 102, 103 is indexed, wherein the index 140 at least indicates in which data object of the multiple data objects 101, 102, 103 a segment 121, 122, 123, ..., 129 is used and the position of a segment 121, 122, 123, ..., 129 on multiple magnetic storage tapes 110, 111, 112, 113, ..., 119, wherein the index 140 is stored on a fast storage medium. In step 340, the processor 150 identifies, based at least on the index 140, a cluster of segments 121, 122, 123, ..., 129 located close to each other on at least one magnetic storage tape 110, 111, 112, 113, ..., 119 of a plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119, wherein the segments in the cluster belong to at least one data object 101, 102, 103 of a plurality of data objects.

[0081] Now refer to Figure 4 , Figure 4 It is used to store cache Figure 1A and Figure 1B103. FIG. 4 is a flow chart of a second method for recovering data on magnetic storage tapes 110, 111, 112, 113, ..., 119 in FIG. In step 410, a cost 160 is calculated for recovering each of the stored data segments 121, 122, 123, ..., 129 stored on the plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119, wherein each of the data segments 121, 122, 123, ..., 129 is part of at least one data object 101, 102, 103. In step 420, based on the calculation, the data segments 121, 122, 123, ..., 129 are sorted from highest recovery cost to lowest recovery cost. At step 430, the data segment with the highest cost is loaded from one of the plurality of magnetic storage tapes 110, 111, 112, 113, ..., 119 into the cache 170, wherein if the remaining available size in the cache 170 is greater than the size of the data segment with the highest cost, the cache 170 is located on the fast storage medium 180. At step 440, the remaining available size in the cache 170 is reduced by the size of the data segment 121, 122, 123, ..., 129 loaded into the fast storage medium.

[0082] Figure 3 and Figure 4 The method shown in the flowchart of can be implemented as a computer program. Such a computer program can be provided on a non-transitory storage medium.

[0083] Although examples are described above, the features and / or steps of these examples may be combined, divided, omitted, rearranged, modified, and / or expanded in any desired manner. Various changes, modifications, and improvements can be readily made by those skilled in the art. Although not expressly described herein, such changes, modifications, and improvements are intended to be part of this specification and are intended to be included within the spirit and scope of the present invention. Therefore, the foregoing description is intended to be illustrative only and not restrictive.

[0084] It is expected that during the life of the patent expiring on this application, many related systems and methods for caching data will be developed, and the scope of the term "caching" is intended a priori to include all such new technologies.

[0085] The terms "including," "having," and variations thereof mean "including but not limited to."

[0086] The term "consisting of" means "including and limited to."

[0087] The term "consisting essentially of" means that a composition, method, or structure may include additional ingredients, steps, and / or parts, but only if the additional ingredients, steps, and / or parts do not materially change the basic and novel characteristics of the claimed composition, method, or structure.

[0088] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. For example, the term "a compound" or "at least one compound" may include a plurality of compounds, including mixtures thereof.

[0089] It will be understood that, for the sake of clarity, certain features of the present invention described in the context of separate embodiments may also be provided in combination in a single embodiment. Conversely, for the sake of brevity, various features of the present invention described in the context of a single embodiment may also be provided individually or in any suitable subcombination or as any suitable other embodiment of the present invention. Certain features described in the context of individual embodiments are not considered essential features of those embodiments, unless the embodiment would be inoperable without those elements.

[0090] Although the present invention has been described in conjunction with the specific embodiments thereof, it is apparent that many alternatives, modifications and variations will be apparent to those skilled in the art. It is therefore intended to encompass all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[0091] The purpose of the applicant is that all publications, patents and patent applications mentioned in this specification are incorporated into this specification by reference in their entirety, as if each individual publication, patent or patent application is specifically and individually pointed out when mentioning to be incorporated into this article by reference. In addition, quoting or identifying any reference of the present application should not be interpreted as admitting that such reference can be used as prior art of the present invention. With regard to using section headings, section headings should not be interpreted as necessary restrictions. In addition, any one or more priority documents of the present application are incorporated into the present application by reference in their entirety.

Claims

1. A system, characterized in that: include: a plurality of magnetic storage tapes, wherein the plurality of magnetic storage tapes have been deduplicated; a plurality of data objects stored on the plurality of magnetic storage tapes, each data object of the plurality of data objects comprising at least two segments, each segment of the at least two segments being stored at at least two different locations on at least one magnetic storage tape of the plurality of magnetic storage tapes; an index, for each segment of the plurality of data objects, the index indicating in which data object of the plurality of data objects the segment is used and a location of the segment on the plurality of magnetic storage tapes, wherein the index is stored on a fast storage medium; A processor for, based at least on the index: identifying a cluster of segments located proximate to one another on at least one of the plurality of magnetic storage tapes, wherein the segments in the cluster belong to at least one of the plurality of data objects; and calculating a cost to restore each stored segment stored on the plurality of magnetic storage tapes, each segment being part of at least one data object; and A cache stores segments based on the costs calculated by the processor, wherein the cache is stored on a fast storage medium.

2. The system according to claim 1, wherein: The cost is based at least in part on an amount of time required to perform a seek to the segment's associated storage segments when restoring a data object to which the segment belongs.

3. The system according to claim 1 or 2, characterized in that Each data object includes in its metadata a user-defined maximum acceptable time for restoring the data object upon request.

4. The system according to any one of claims 1 to 3, characterized in that Each segment includes in its metadata a weight based on the number of data objects that use the segment.

5. The system according to claim 4, characterized in that If the weight exceeds a given threshold, the data object is loaded into the cache.

6. The system according to any one of claims 1 to 5, characterized in that Each segment includes a fingerprint in its metadata, which comprises a strong hash value.

7. The system according to any one of claims 1 to 6, characterized in that The fast storage medium includes a plurality of solid state hard drives.

8. The system according to any one of claims 1 to 7, characterized in that The plurality of magnetic storage tapes are included in one or more deduplication domains.

9. The system according to any one of claims 1 to 8, characterized in that The at least two segments are stored on one or more magnetic storage tapes.

10. A method, characterized in that include: Deduplication of data across multiple magnetic storage tapes; storing a plurality of data objects on the plurality of magnetic storage tapes, each data object of the plurality of data objects comprising at least two segments, each segment of the at least two segments being stored at at least two different locations on at least one magnetic storage tape of the plurality of magnetic storage tapes; indexing each segment of the plurality of data objects, the index indicating at least in which data object of the plurality of data objects the segment is used and a location of the segment on the plurality of magnetic storage tapes, wherein the index is stored on a fast storage medium; a processor identifying, based at least on the index, a cluster of segments located proximate to one another on at least one of the plurality of magnetic storage tapes, the segments in the cluster belonging to at least one of the plurality of data objects; The processor identifies a segment used by more than one data object; calculating a cost to restore each stored segment stored on the plurality of magnetic storage tapes, each segment being part of at least one data object; and The segments are stored in a cache based on the costs, wherein the cache is stored on a fast storage medium.

11. The method according to claim 10, characterized in that The cost is based at least in part on an amount of time required to perform a seek to an associated storage segment of the segment when restoring a data object to which the segment belongs.

12. The method according to claim 10 or 11, characterized in that Each data object includes in its metadata a user-defined maximum acceptable time for restoring the data object upon request.

13. The method according to claim 11 or 12, characterized in that Each segment includes in its metadata a weight based on the number of data objects that use the segment.

14. The method according to claim 12 or 13, characterized in that If the weight exceeds a given threshold, the data object is loaded into the cache.

15. The method according to any one of claims 10 to 14, characterized in that Each segment is fingerprinted in its metadata, which includes a strong hash value.

16. The method according to any one of claims 10 to 15, characterized in that The fast storage medium includes a plurality of solid state hard drives.

17. The method according to any one of claims 10 to 16, characterized in that The plurality of magnetic storage tapes are included in one or more deduplication domains.

18. The system according to any one of claims 10 to 17, characterized in that The at least two segments are stored on one or more magnetic storage tapes.

19. A computer program for caching data objects stored on a magnetic storage tape, characterized in that The computer program comprises program instructions which, when executed by at least one processor, cause the at least one processor to: Deduplication of data across multiple magnetic storage tapes; storing a plurality of data objects on the plurality of magnetic storage tapes, each data object of the plurality of data objects comprising at least two segments, each segment of the at least two segments being stored at at least two different locations on the plurality of magnetic storage tapes; indexing each segment of the plurality of data objects, the index indicating at least in which data object of the plurality of data objects the segment is used and a location of the segment on the plurality of magnetic storage tapes, wherein the index is stored on a fast storage medium; a processor identifying, based at least on the index, a cluster of segments located proximate to one another on at least one of the plurality of magnetic storage tapes, the segments in the cluster belonging to at least one of the plurality of data objects; The processor identifies a segment used by more than one data object; calculating a cost to restore each stored segment stored on the plurality of magnetic storage tapes, each segment being part of at least one data object; and The segment is stored in a cache based on the determination by the processor, wherein the cache is stored on a fast storage medium.

20. A system, characterized in that: include: multiple magnetic storage tapes; a plurality of data objects stored on the plurality of magnetic storage tapes, each data object of the plurality of data objects comprising at least two segments, each segment of the at least two segments being stored at at least two different locations on the plurality of magnetic storage tapes; Processor for: calculating a cost of recovering each stored data segment stored on the plurality of magnetic storage tapes, each data segment being part of at least one data object; Based on the calculation, sorting the data segments from highest recovery cost to lowest recovery cost; loading a highest-cost data segment from one of the plurality of magnetic storage tapes to a cache, wherein the cache is located on a fast storage medium if a remaining available size in the cache is greater than a size of the highest-cost data segment; and The remaining available size in the cache is reduced by the size of the data object segment loaded to the fast storage medium.

21. The system according to claim 20, wherein: The cost is calculated based on at least the commonality of each stored data segment of a plurality of data objects including the at least one data object, a time required to recover the stored data segment from a magnetic storage tape on which the stored data segment is stored, and a service level agreement including a user-defined maximum acceptable time for recovering the data object upon request.

22. The system according to claims 20 to 21, characterized in that The cost is weighted to take into account the number of data objects using the segment.

23. The system according to any one of claims 20 to 22, characterized in that Each segment includes a fingerprint in its metadata, which comprises a strong hash value.

24. The system according to any one of claims 20 to 23, characterized in that The fast storage medium includes a plurality of solid state hard drives.

25. The system according to any one of claims 20 to 24, characterized in that The plurality of magnetic storage tapes have been deduplicated in one or more deduplication domains.

26. A method, characterized in that include: calculating a cost of recovering each stored data segment stored on a plurality of magnetic storage tapes, each data segment being part of at least one data object; Based on the calculation, sorting the data segments from highest recovery cost to lowest recovery cost; loading a highest-cost data segment from one of the plurality of magnetic storage tapes to a cache, wherein the cache is located on a fast storage medium if a remaining available size in the cache is greater than a size of the highest-cost data segment; and The remaining available size in the cache is reduced by the size of the data segment loaded to the fast storage medium.

27. The method according to claim 26, characterized in that The cost is calculated based on at least the commonality of each stored data segment of a plurality of data objects including the at least one data object, a time required to recover the stored data segment from a magnetic storage tape on which the stored data segment is stored, and a service level agreement including a user-defined maximum acceptable time for recovering the data object.

28. The method according to claims 26 to 27, characterized in that The cost is weighted to take into account the number of data objects using the segment.

29. The method according to any one of claims 26 to 28, characterized in that Each segment is fingerprinted in its metadata, which includes a strong hash value.

30. The method according to any one of claims 26 to 29, characterized in that The fast storage medium includes a plurality of solid state hard drives.

31. A computer program for caching data objects stored on a magnetic storage tape, characterized in that The computer program comprises program instructions which, when executed by at least one processor, cause the at least one processor to: calculating a cost of recovering each stored data segment stored on a plurality of magnetic storage tapes, each data segment being part of at least one data object; Based on the calculation, sorting the data segments from highest recovery cost to lowest recovery cost; loading a highest-cost data segment from one of the plurality of magnetic storage tapes to a cache, wherein the cache is located on a fast storage medium if a remaining available size in the cache is greater than a size of the highest-cost data segment; and The remaining available size in the cache is reduced by the size of the data segment loaded to the fast storage medium.

Images