Cache indexing using fingerprint-based data addresses

Abstract

A cache storage system indexing system is provided that indexes data addresses in the cache storage system based on a data fingerprint of the cached data, the data fingerprint being generated by a deduplication fingerprint function used to reference deduplication of the data in the cache storage system. The system includes obtaining a data fingerprint for the data of a data operation by either applying the deduplication fingerprint function to the data of a write operation or accessing deduplication metadata of a read operation to obtain a data fingerprint generated by using the deduplication fingerprint function used to deduplicate the data in the cache storage system, and using an indexing service to the cache storage system having an address schema based on the data fingerprint of the data.

Description

Technical Field

[0001] The present invention relates to cache storage, and more specifically, to cache indexing using a data address based on a data fingerprint.

Background Art

[0002] Cache storage is a physical memory space used to temporarily store data for handling read and write operations of data in order to provide high performance. Data is stored in the cache as a result of previous operations as an intermediary to back-end storage such as disk storage. Data access from the cache is faster than access from the back-end storage, and the data is stored in the cache while being stored in the back-end storage by a write-through or write-back operation.

[0003] Cache sharding is a method for allocating data items to nodes of a distributed cache system based on a hash function calculated from an item identifier. Sharding horizontally scales the cache system to accommodate processing power and storage capacity. A set of data items is partitioned into "shards" based on the result of a hash function calculated based on the identifier of the data item. Each shard is mapped to a cache device to partition data among members of the cluster and to identify the member of the cluster responsible for a given data item by calculating the hash function.

[0004] Identifiers used in known cache sharding have traditionally been customer identifiers or host identifiers, such as Internet Protocol addresses, or other resource identifiers, such as central processing unit (CPU) or CPU core identifiers. The typical purpose is to distribute the workload across many resources, such as the CPU. In particular, in scenarios where very few logical volumes are used, the system still utilizes all available resources and does not become bottlenecked by a single resource, even though a single logical volume is being used.

[0005] Cache deduplication is a known method for reducing the number of copies of data blocks stored in a cache in order to reduce cache usage and storage capacity requirements. Deduplication works at the block level, reducing duplicate data blocks and remembering the source data blocks to which other data blocks are mapped. When data is written to the cache, the deduplication service scans the incoming block, creates a fingerprint using a hash function, and stores the fingerprint in deduplication metadata in the form of a hash store-in-memory data structure. Large-scale cache deduplication with 100% data coverage is costly due to the size of the data structure for the required deduplication metadata and the time it takes to look it up.

[0006] Traditional cache lookups use virtual address or logical block address (LBA) mapping to cache addresses. This means that data is cached and accessible for a specific logical address. As a result, access to the cache is limited to a single host accessing a specific logical address and cached data, as defined at the software layer.

[0007] The problem with current technology is that traditional LBA addressing requires multiple instances of the cache as reads / writes traverse various layers within the storage subsystem. Each layer represents a different context within the system, each with its own index. This necessitates a significant amount of resources and redundant effort to manage multiple caches. [Overview of the project]

[0008] According to one aspect of the present invention, a cache storage system indexing method is provided, which includes indexing data addresses in a cache storage system based on a data fingerprint of cached data, wherein the data fingerprint is generated by a deduplication fingerprinting function used to refer to the deduplication of data in the cache storage system.

[0009] Data segment deduplication fingerprinting provides a uniform method for indexing data segments, independent of the context in which the data is placed in the cache service, and enables the sharing of the cache service across all logical addressing schemes used in different layers of storage within the storage controller to speed up reads. This method achieves faster average reads with a more unified cache service while maintaining the existing isolation for traditional addressed cache writes.

[0010] Data address indexing can be based on a subset of data fingerprints as references to arrays that store cache addresses in the cache address space of a cache storage system. In one embodiment, the subset of data fingerprints is obtained as an absolute intermediate subset of data fingerprints of a defined number of bytes modulo the size of the array.

[0011] Data fingerprints may be obtained from read operations from deduplication metadata, mapping different address aliases to deduplication data fingerprints from different software logical layers or different client domains. Cache indexing methods can be used in addition to, or instead of, the original cache addressing method.

[0012] The cache storage system may be a single uniform cache for read / write operations across multiple layers of the storage subsystem, and indexing provides uniform indexing across multiple layers.

[0013] The cache storage system may be a sharded cache with sharding based on data fingerprints generated by a deduplication fingerprinting function.

[0014] According to another aspect of the present invention, a computer implementation method for data operations on a cache storage system is provided, which includes obtaining a data fingerprint of data for a data operation by either applying a deduplication fingerprinting function to data in a write operation or accessing deduplication metadata for a read operation, in order to obtain a data fingerprint generated by using a deduplication fingerprinting function used for deduplication of data in the cache storage system, and using an indexing service to a cache storage system having an address schema based on the data fingerprint of the data.

[0015] The indexing service can use references based on a subset of data fingerprints as addresses to arrays that store cache addresses in the cache address space of the cache storage system.

[0016] If the data operation is a write operation, the method may include receiving a data write operation, calculating a data fingerprint of the data to be written using a fingerprint function used for data deduplication in a cache storage system, creating a cache entry for the data to be written, and using the data fingerprint to address the cache entry in the cache storage system.

[0017] If the data operation is a read operation, the method may include receiving a data read operation having a logical address for the data, obtaining a data fingerprint of the data to be read from the deduplication metadata by referring to the logical address, and using the data fingerprint to address the data in the cache storage system from which the data is to be read.

[0018] Since data fingerprinting could not be performed using logical addresses, it may be performed after a cache miss using the original addressing scheme of the cache storage system when the read operation comes down to the deduplication layer. Obtaining data fingerprinting and using data fingerprinting to address the data may be performed before or in parallel with the original addressing scheme.

[0019] This method may include enabling false hits in a cache storage system that uses data fingerprints to address data, and matching the data fingerprints with the data fingerprints of the retrieved data.

[0020] This method may include using a subset of data fingerprints as addresses to arrays that store cache addresses, and matching the data fingerprints with the data fingerprints of the retrieved data.

[0021] This method may further include discovering multiple references to a data source by using data fingerprint addressing, and unifying these references to the data source, thereby increasing deduplication.

[0022] As the read operation passes through the software layer, this method can record an address alias resulting from address translation to refer to the same data segment, and can associate the address alias with a data fingerprint.

[0023] According to another aspect of the present invention, a cache storage system is provided which includes an indexing system for a cache storage system which includes an array for storing cache addresses in a shared cache address space, the array having references based on data fingerprints of cached data, the data fingerprints being generated by a deduplication fingerprinting function used for deduplication of data in the cache storage system.

[0024] The indexing system for a cache storage system may include a data fingerprinting component for obtaining a deduplication data fingerprint when performing read and write operations on the cache storage system. The data fingerprinting component is a read operation component for obtaining a data fingerprint for a read operation from deduplication metadata, the deduplication metadata may include a read operation component that maps address aliases from different software logical layers or different client domains to a deduplication data fingerprint, and a write operation component for applying the deduplication fingerprinting functionality to the data of a write operation.

[0025] A cache storage system may include a single uniform cache for read / write operations across multiple layers of the storage subsystem, and the cache storage system may provide uniform indexing across multiple layers. The cache storage system may be a sharded cache with sharding based on data fingerprints generated by a deduplication fingerprinting function.

[0026] According to another aspect of the present invention, an indexing system for a cache storage system is provided, comprising: a data fingerprint acquisition component for acquiring a data fingerprint for data of a data operation by either applying a deduplication fingerprint function to data of a write operation or accessing deduplication metadata for a read operation, in order to acquire a data fingerprint generated by using a deduplication fingerprint function used for deduplication of data in the cache storage system; and an array lookup component for acquiring a cache address for data of data of a data operation to the cache storage system by using a reference based on the data fingerprint of the data.

[0027] The indexing system of a cache storage system may include a write operation component for receiving a data write operation, calculating a data fingerprint of the data to be written using a fingerprint function used for data deduplication within the cache storage system, creating a cache entry for the data to be written, and using the data fingerprint to address the cache entry within the cache storage system.

[0028] The indexing system of a cache storage system can include a read operation component for receiving a data read operation having a logical address for data, obtaining a data fingerprint of the data to be read from the deduplication metadata by referring to the logical address, and using the data fingerprint to address data within the cache storage system for which the data is to be read.

[0029] The indexing system of a cache storage system can include a hit matching component for matching a data fingerprint with the data fingerprint of the retrieved data.

[0030] The indexing system of a cache storage system may include a reference unification component for discovering multiple references to a data source by using data fingerprint addressing, unifying the references to the data source, thereby increasing deduplication.

[0031] According to a further aspect of the present invention, a sharded cache storage system is provided that uses a data fingerprint address as a shard key, the data fingerprint address being an address based on a data fingerprint obtained using a deduplication fingerprinting function used for deduplication of data within the cache storage system.

[0032] This has the advantage of evenly and fairly distributing the responsibility for fingerprints to various resources (such as CPU cores) within the system, so that on average the various resources process fingerprints equally and the risk of skew is minimized. This also helps in segmenting the cache in error scenarios.

[0033] A further aspect of the present invention provides a computer program product for indexing a cache storage system, the computer program product comprising a computer-readable storage medium having program instructions implemented therein, the program instructions being executable by a processor, causing the processor to perform the following: obtain a data fingerprint of data for a data operation by either applying a deduplication fingerprinting function to the data of a write operation or accessing deduplication metadata for a read operation, in order to obtain a data fingerprint generated by using a deduplication fingerprinting function used for deduplication of data in the cache storage system; and use an indexing service to a cache storage system having an address schema based on the data fingerprint of the data.

[0034] The computer-readable storage medium may be a non-temporary computer-readable storage medium, and the computer-readable program code may be executable by a processing circuit.

[0035] The subject matter considered to be part of the present invention is specifically pointed out and explicitly asserted in the concluding section of this specification. The present invention, with respect to both the organization and the method of operation, along with its objectives, features, and advantages, can be best understood by referring to the following detailed description, when read together with the accompanying drawings.

[0036] Next, preferred embodiments of the present invention will be described with reference to the following drawings for illustrative purposes only. [Brief explanation of the drawing]

[0037] [Figure 1A] This is a schematic diagram illustrating an embodiment of a caching system including a caching indexing system according to the present invention. [Figure 1B] This is a schematic diagram illustrating an embodiment of a caching system including a caching indexing system according to the present invention. [Figure 2A]This is a schematic diagram illustrating the software layer address used according to the present invention. [Figure 2B] This is a schematic diagram of a clustered storage controller used in the implementation of the system according to the present invention. [Figure 3] This is a flowchart of an example of an embodiment of one aspect of the writing operation method according to the present invention. [Figure 4A] This is a flowchart illustrating an example of another embodiment of the reading operation method according to the present invention. [Figure 4B] This is a flowchart of another exemplary embodiment of one aspect of the reading operation method according to the present invention. [Figure 5] This is a block diagram of one embodiment of the system according to the present invention. [Figure 6] This is a block diagram of one embodiment of a computer system or cloud server in which the present invention may be implemented. [Figure 7] This is a schematic diagram of a cloud computing environment in which the present invention may be implemented. [Figure 8] This is a diagram of an abstraction model layer of a cloud computing environment in which the present invention may be implemented.

[0038] For the sake of simplification and clarity in the illustrations, it should be understood that the elements depicted are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Furthermore, where appropriate, reference numbers may be repeated between drawings to indicate corresponding or similar features. [Modes for carrying out the invention]

[0039] A method and system for providing an indexing method for a unified cache address scheme based on data fingerprints generated by a hash function used for data deduplication lookups is described. Given that deduplication is becoming an increasingly common feature in storage systems, creating a single unified cache across multiple layers using deduplication metadata significantly optimizes the storage system by reducing the cache to a single instance. Cache sharding may also be based on deduplication fingerprints.

[0040] The term "deduplicated fingerprint" is used to refer to the hash obtained from the hash function used to deduplicate data. This distinguishes the deduplicated hash from the hash of identifiers used in known cache sharding.

[0041] The described methods and systems can cache data to provide and utilize a mechanism for combining data addressed from different software logical layers and different client domains, possibly data accessed using different addressing schemes and namespaces, into a unified cache addressing scheme based on a common fingerprint function. A deduplicated fingerprint, computed for writes and retrieved from deduplicated metadata for reads, is used to address the cache service as a common cache service unifying client domains. The cache service may be partitioned into shards based on the deduplicated fingerprint. This can be used in conjunction with, or independently of, cache indexing and addressing using the deduplicated fingerprint.

[0042] Deduplication is a well-known technique used to remove duplicates from stored data blocks by using deduplication metadata to point to the stored blocks. The deduplication process begins by creating a unique digital fingerprint or signature (often called a hash) for a given data block by applying a hash function. This fingerprint value is stored in a fingerprint database or hash store so that it can be compared to the fingerprint value created for a newly received data block. By comparing the fingerprint values, it is determined whether the data block is unique or a duplicate of a data block that has already been stored. Deduplication metadata provides a mapping from the deduplication fingerprint to the address of the stored data block. Deduplication fingerprint values ​​are used in the indexing methods and systems for the data caching services described.

[0043] The described method uses deduplication fingerprints to map to cache addresses and checks whether the data exists in the cache. This can be used separately from, or as an alternative or additional form of, traditional lookups that use virtual addresses or logical block addresses (LBAs) mapped to cache addresses.

[0044] Caches can also be sharded using deduplication fingerprints for cache partitioning purposes. By partitioning data into different subsets based on deduplication fingerprints and deduplication them in different logical or physical locations, sharded data improves coverage and cost balance. For example, if a cache is partitioned by a backend storage controller, a failure in one controller will not affect other cache partitions or seize resources. Sharding allows for an equal and fair distribution of fingerprint responsibility among various resources in the system (e.g., CPU cores), ensuring that on average, different resources handle fingerprints equally and minimizing the risk of skew.

[0045] The indexing method can be used in any caching implementation where a caching service is required, allowing multiple users who need caching to share the same service. Since they all share the cache, all users can benefit from each other's data.

[0046] This can be implemented in any area where caching services and deduplication technologies exist. These may reside in the physical drives, storage controllers, or even the host server itself.

[0047] The described methods and systems improve the read cache hit rate by combining data from more cache users in an equal or more relevant manner within the cache through cache sharding. This allows for generating read "cache hits" for data that has not been recently read or written, or possibly ever, using addresses used in read operations.

[0048] The addressing scheme can be used independently or in parallel with the original addressing scheme. If a cache miss occurs using the original addressing scheme for a read operation, the deduplication metadata is used to resolve the stored deduplication fingerprint of the data without reading the actual user data, and that deduplication fingerprint is then used for addressing in the unified cache.

[0049] Referring to Figure 1A, schematic diagram 100 shows an exemplary embodiment of the system described for the cache indexing system 110 for the cache service 120. The cache service 120 may be sharded (140) and may include a plurality of cache nodes 121-124.

[0050] The described deduplication-based cache indexing system 110 refers to the deduplication metadata 150 of the cache service 120 and uses the deduplication fingerprint 151 of the data being read or written as the basis for the cache address 111 and, optionally, as the basis for the sharding 130 of the cache service 120.

[0051] Note that in the described system and method, the cache service 120 is not deduplicated. Cache deduplicating may be performed by another service, and a typical deduplicated cache would have indexing means for looking up the source address of deduplicated data during write operations. Deduplication metadata 150 provided by such a deduplicating service is referenced by the described system and method.

[0052] The deduplication-based cache indexing system 110 indexes cache entries from the deduplication fingerprint 151. As described above, a typical deduplication cache has indexing means for looking up the source address of the deduplication data during a write operation, but this would not enable the addressing described herein, which is applicable to both write and read operations. The described cache uses indexing from the deduplication fingerprint 151 to cache entries, using the described indexing when reading and adding cache entries of the deduplication fingerprint 151 when writing.

[0053] The described read indexing may be used to perform a primary addressing function. Alternatively, it may be used to perform a secondary "double-take" of the cache service 120 by mapping the original addressing scheme, which may be a conventional addressing scheme, to a deduplication-based addressing scheme.

[0054] The secondary addressing method is shown in Figure 1A, where the conventional address is shown as "Addr" 161, which receives a "cache miss" 101 from the cache service 120. The conventional address 161 is used to look up the deduplication metadata 150 to resolve the data deduplication fingerprint 151 without retrieving the data content (102). This is then used by the cache indexing system 110 to retrieve the cache address 111 based on the deduplication fingerprint and perform a secondary pass 103 over the cached data.

[0055] The fact that, during a read, the deduplication fingerprint 151 for a portion of the data provides a uniform way of indexing data segments that are independent of the context in which the data was placed in the cache service 120 allows the cache service 120 to be shared across all logical addressing schemes used in different layers of storage within the storage controller to speed up reads. This is indicated by address aliases 162-164 of the conventional address "Addr" 161, which can be used as an alternative to look up the deduplication metadata 150 to obtain the deduplication fingerprint 151.

[0056] In the primary addressing method, the cache miss step 101 is not performed, and the deduplication fingerprint address 111 is used as the primary address for the cache service 120.

[0057] Referring to Figure 1B, schematic Figure 105 shows an exemplary embodiment with further details of the mapping provided by the deduplication metadata 150. For data write deduplication, the deduplication metadata 150 is stored containing an index 152 of the deduplication fingerprint 151 generated by a hash function for each block of data. The deduplication metadata 150 contains the source addresses of logical addresses 161, 171, 181. 153 This also includes a many-to-one mapping to the source address, where logical addresses 161, 171, and 181 are the source addresses. 153This is for data blocks that are copies of data blocks stored in [location]. Logical addresses can use logical addresses 161-163, 171-173, and 181-183 of the logical addressing scheme used in different software layers.

[0058] In the described method and system, the deduplication-based cache indexing system 110 uses deduplication metadata 150 to map logical addresses 161-163, 171-173, and 181-183 for read operations to a deduplication fingerprint 151 used for address 111 of the cache service 120. This deduplication-based cache indexing uses the source address 153 The original cache indexing system 112, which uses a cache addressing scheme 113 based on this, may be used independently of or in parallel with it.

[0059] Figure 2A shows a schematic diagram 220 of different software layers 221-224. In exemplary embodiments, layer 1 221 may be the upper cache, layer 2 222 the deduplication layer, layer 3 223 the lower cache, and layer 4 224 the RAID layer. Each layer is optional, and I / O operations may bypass that layer and flow to the layer directly below it. In some embodiments, there may be more layers between layer 1 221 and layer 2 222, for example, an additional layer may be a replication service for performing volume mirroring, and another additional layer may be a point-in-time copy service for providing a point-in-time snapshot service. Furthermore, layer 4 224 may include a virtualization layer in which multiple RAID arrays are all virtualized on the same host volume. The RAID layer itself may include, for example, a cache for managing RAID parity.

[0060] The read / write operation 230 flows through layers, some of which are optional, and the read / write operation 230 may bypass that layer and flow to the layer directly below it. All layers may require access to the described cache indexing service. When the read or write operation 230 travels through software layers 221-224, if the calculation results in an address translation from the original address 231 to refer to the same data segment, the selected address "aliases" 232-234 remain associated with the operation 230.

[0061] When the deduplication layer metadata stores the deduplication fingerprint associated with address 231, address aliases 232-234 can be additionally associated with the deduplication fingerprint and subsequently used for retrieval.

[0062] After the metadata of the lower-level deduplication layer resolves the deduplication fingerprint 151, the deduplication-based cache indexing system 110 can access the full picture of the cache content using the deduplication fingerprint 151 without yet retrieving the data content.

[0063] Figure 2B is a schematic diagram 200 of a clustered cache service 210 with nodes 211-216 that client computers 201 and 202 access to storage. Nodes 211-216 are provided in pairs to provide high availability. Clustering can increase the availability of the cache service, so that even if a node fails, the rest of the cache remains accessible. Clustering is frequently used in combination with replication and failover. Each node is replicable, and if a node fails, a replica can be quickly brought online.

[0064] Indexing a cache from a deduplication fingerprint of the requested data is aided by the fingerprint's attributes. Deduplication fingerprints, such as SHA hash codes, are randomly distributed, making them an excellent basis for very fast hash maps, and this applies to any subset of the SHA bits. This means that the indexing method can truncate or select subsets of the SHA and use them as addresses for arrays that store cache addresses (or nulls) in the shared cache address space.

[0065] In an exemplary implementation, the following indexing and addressing based on deduplicated fingerprints can be used. A typical deduplicated fingerprint would be a 20-byte hash of Secure Hash Algorithm 1 (SHA-1). Other SHA schemes exist, such as SHA-2, but these have larger hashes. Creating a table using a 20-byte hash results in a very large table index, thus consuming a high level of resources. Therefore, a subset of the bytes of the deduplicated fingerprint can be used to create an indexing table.

[0066] In an exemplary embodiment, the central 6 bytes of the hash can be used to create an indexing table. The following sum is performed, and the resulting value can be used to represent the hash entry in the cache indexing table. ((absolute value of the middle 6 bytes of the hash) modulo(selected table size)) Here, "modulo" means dividing X by Y and taking the remainder of the sum.

[0067] For example, if the middle 6 bytes of the fingerprint represent 0x123456, and the table size is 0x10000 entries, then the table entries for the cache index would be: 0x123456% 0x10000 = 0x3456.

[0068] This method provides a random distribution of deduplicated fingerprints on the cache indexing table.

[0069] This is just one example; in another example, you could take the middle 6 bytes of a 20-byte fingerprint hash and create a tree (instead of a table) of those hashes.

[0070] Another useful attribute of deduplication fingerprints when indexing caches is that they allow indexing to return false "hits" because there is fast hardware support to compute the SHA of the data pointed to by the hash map to ensure an exact match. Since there may be multiple entries in the same hash chain within the cache indexing table, a full hash is needed to determine which entry to use in the chain of partial hashes.

[0071] This is further illustrated with an implementation example that uses the middle 6 bytes of a 20-byte hash. A 20-byte hash offers higher data precision compared to a 6-byte hash, which can result in hash collisions (i.e., two different sets of data with the same 6-byte hash). Therefore, it may be necessary to use the full 20-byte hash to return the correct dataset.

[0072] The use of the central 6 bytes can also be used to reduce the size of the deduplication metadata by storing only partial hints. The full hash is stored, but there is a deduplication metadata cache for fast lookups of the deduplication metadata.

[0073] Referring to Figure 3, flowchart 300 shows an exemplary embodiment of a method for cache indexing based on deduplication fingerprints during a write operation.

[0074] A receive write operation is received that has a logical address for the data (301). The method may branch to perform a write to the backend and complete a write to the client (302), and the method branch may terminate (308).

[0075] In parallel with writing to a client, a branch can cache the write data. A cached write operation may include calculating a deduplication fingerprint of the data using a hash function as part of the deduplication write process (303). Typically, a data write operation obtains a deduplication fingerprint (also called a hash code or SHA) of the data because the write has data to hash, and the write operation then uses the fingerprint to find duplicates. The data write then creates and maintains metadata of the duplicates found as a many-to-one reference-source link that can be used for subsequent reading and writing of the data to the shared source.

[0076] A cache branch may look up a deduplicated fingerprint for the data in the cache (304). If a deduplicated fingerprint is found in the cache (305), no action may be taken and the branch may terminate (308). A deduplicated fingerprint match ensures that the data to be written is the same as the cached data. This can be matched if the lookup is performed on a subset of the deduplicated fingerprints.

[0077] However, if the deduplicated fingerprint is not found in the cache, a cache entry is created for the deduplicated fingerprint (306), the data is cached (307), and the method branch may terminate (308).

[0078] Referring to Figure 4A, Flowchart 400 shows an exemplary embodiment of a method for cache indexing based on deduplicated fingerprints during a read operation.

[0079] A receive read operation is received that has a logical address for the data (401). The logical address may be a logical address in a logical addressing scheme used in one of the software layers.

[0080] A typical data read cannot calculate the associated deduplication fingerprint because it does not yet know the content of the requested data. The data read uses logical addresses that are processed to map to physical (i.e., next-layer) addresses, which are then converted to the equivalent addresses used for non-deduplication data, or to the shared source addresses of the deduplication data.

[0081] The described method allows access to and lookup (402) deduplication metadata maintained for data deduplication. The lookup 402 retrieves the deduplication fingerprint indexed in the deduplication metadata for the data at the logical address.

[0082] As mentioned earlier, deduplication metadata typically stores a deduplication fingerprint of a data segment for write operations, but this has not traditionally been used for reads (apart from its use as additional data matching in some schemes). The method described utilizes the fact that when a read accesses the deduplication metadata to obtain the physical address, it can also obtain an additional deduplication fingerprint of the data to be read. This uses the deduplication metadata in the backing store to provide address translation from the read LBA address to the deduplication fingerprint of the data content. This uses the deduplication metadata to obtain the deduplication fingerprint of the requested data before the data itself is retrieved in the read operation. Deduplication metadata is often stored in memory or a faster storage layer and is looked up anyway to determine the physical address.

[0083] The cache service reads cache addresses using an indexing method based on deduplicate fingerprints (403). The indexing method may truncate or select a subset of deduplicate fingerprints and use them as addresses to an array that stores cache addresses (or nulls) in the shared cache address space.

[0084] A cache hit occurs, and it is determined whether the data was found (404). If the data is not found, a cache miss occurs (409), and the data may be read from backend storage. If a subset of the deduplicated fingerprint is used as the address, a false hit is possible, and the entire deduplicated fingerprint may be used to match the correct location of the cached data for reading (405).

[0085] The system determines whether a match exists (406). If there is no match, a cache miss occurs (409), and the data may be read from backend storage. If a match exists, the data may be read from the cache (407).

[0086] Therefore, a hit in the form of a non-null entry from the cache's deduplication fingerprint to the address map may allow both retrieving data from the cache and discovering another reference to the source. This can be used to further unify two references to one source, or two sources with multiple references (408), thereby potentially increasing deduplication.

[0087] Deduplication metadata may indicate that the data source is split into two. The most likely reason for this is that the scope of the deduplication domain does not cover the original lookup of the hash when the second write occurred, and therefore it could not discover that the data was already stored. The described cache indexing scheme can expand the scope of the deduplication domain, and as a result, the new data source can be discovered. This allows writes from the second source to be effectively replaced with links to writes from the first source, saving capacity from the user's perspective.

[0088] Furthermore, the described method can enable increased data deduplication compared to existing schemes for either backend storage or cache deduplication. To illustrate this, please refer to Figure 2A, which illustrates a typical tiering scheme of the various components of a system. Given that I / O flows from top to bottom, components below the deduplication layer may be generating data that the deduplication layer cannot access. This is one reason why this feature may save deduplication capacity. For example, consider a copy service layer or snapshot layer (copying one volume to another) below the deduplication layer. The copy service layer will generate new writes that are not exposed by the deduplication layer, thus increasing capacity usage. The described method can save capacity by utilizing the cache as a source for identifying new deduplication.

[0089] The indexing method described takes into account that deduplication is typically not 100% complete in storage controllers and adds an alternative path to read operations. Complete deduplication is usually too costly in terms of the required metadata size and search performance, but improving the deduplication rate is highly desirable.

[0090] Deduplication degrades performance within the storage controller. Depending on the I / O workload, especially if there is a high percentage of duplicate data writes, the storage controller may choose to prioritize performance at the expense of capacity reduction. For example, if there are many volumes writing the same data, numerous parallel accesses to the deduplication metadata occur, and the metadata blocks are updated frequently, requiring internal locks on the metadata and degrading performance. In some situations, the controller may choose to create separate sets of the same source data to avoid sacrificing performance. The controller can then reintegrate multiple data sources into a single source, which can be done asynchronously with respect to the main I / O flow. Cache indexing schemes can be used to reintegrate multiple sources.

[0091] The ability to obtain a deduplication fingerprint is used as an append or replacement index in the cache. In one embodiment, a read operation may descend to the deduplication layer only if it cannot be served using a logical address, starting from a cache layer above the deduplication layer. The I / O operation can then use the deduplication fingerprint to make a second request to the cache service, requesting whether data with a matching deduplication fingerprint exists in the cache.

[0092] Referring to Figure 4B, flowchart 450 provides a more detailed exemplary embodiment of the reading operation.

[0093] A receive read operation is received along with a logical address (451). The branching method allows for a lookup of data in the conventional cache (452). If the data is found (453), it can be retrieved from the conventional cache (454); otherwise, there is a cache miss.

[0094] Another branch of the method is that the logical address of the read data can be used to look up the deduplication fingerprint in the deduplication metadata of the read data (455). If the deduplication fingerprint is not found (456), the data may be read from the backend (457). If the deduplication fingerprint is found (456), the method can look up the data in the deduplication-based cache (458), and if found (459), retrieve the data from the deduplication-based cache (460). If the data is not found (459), the data may be read from the backend (461).

[0095] This branching operation is either performed prior to or concurrently with the conventional reads used by lower layers or backend devices to retrieve data. If performed concurrently, cache misses will not delay the read. If both searches are performed concurrently, the conventional search is faster, allowing the lookup of deduplication metadata to be aborted afterward.

[0096] This method can complete the read to the client (462) in parallel with checking the data source (463). If it is determined that the data has not been read from the deduplication cache (464), a deduplication cache data entry is created based on the deduplication fingerprint address (465) and the data is cached (466). Otherwise, the method terminates (470).

[0097] <Cache sharding using deduplication fingerprinting> Cache partitioning occurs primarily for two reasons. First, different layers or operations may attempt to use the cache but use different addressing schemes. Second, caches are often partitioned to allow for a certain level of isolation or partitioning, preventing cache "leaks" due to performance degradation of a subset of underlying storage media, such as when reads targeting backups within the cache occur by slower drives.

[0098] Horizontal partitioning (often called sharding) is a strategy where each partition is a data store in itself, but all partitions share the same schema. Each partition is called a shard and holds a specific subset of the data.

[0099] A sharding scheme based on deduplication fingerprints allows for the "unification" of the cache across both dimensions in read operations where the deduplication fingerprint can be retrieved faster than the contents of the data segment. This logically increases the effective size of the cache used, but incurs the additional cost that the data is not cached in the addressing scheme used for each of these reads.

[0100] A cache has a first horizontal dimension, using all resources (partitions and CPU cores, or other resources) at a specific layer, and a second vertical dimension, with separate layers in the I / O stack architecture. Therefore, other layers benefit from the cache. Generally, I / O in a stack architecture only flows from top to bottom (completion of I / O goes up the stack), so components of the I / O stack will only benefit from the cache if they are below that layer. One advantage of the described index scheme is that all layers benefit from the cache regardless of their position in the stack, because hash lookups can be performed using deduplication metadata and fingerprints.

[0101] Referring to Figure 5, this is a block diagram of an exemplary embodiment of a computing system 500 as part of a storage system, which provides a cache storage system indexing system 510 in the form of a deduplication-based cache indexing system 110 for cache storage 550.

[0102] The computing system 500 may include an original cache indexing system 530 for cache storage 550, which includes an original address array 531. The original cache indexing system 530 may be used in parallel with or in succession to a deduplication-based cache indexing system 110.

[0103] Furthermore, the computer system 500 includes a deduplication system 540 which includes stored deduplication metadata 541.

[0104] The cache storage 550 may optionally be sharded as described above with partitioning using data fingerprint addresses as shard keys, where the data fingerprint addresses are addresses based on data fingerprints obtained using a deduplication fingerprint function used for data deduplication in the cache storage system.

[0105] The computing system 500 includes circuitry for performing the functions of the described components, which may be at least one processor 501, hardware modules, or software units running on at least one processor. Multiple processors are provided to execute parallel processing threads, enabling parallel processing of some or all of the components' functions. Memory 502 may be configured to provide computer instructions 503 for performing the components' functions to at least one processor 501.

[0106] The cache storage system indexing system 510 includes a deduplication-based address array 514 that stores cache addresses in a shared cache address space, the array having references based on data fingerprints of cached data, the data fingerprints being generated by a deduplication fingerprinting function used for deduplication of data within the cache storage system.

[0107] The cache storage system indexing system 510 also includes a data fingerprint acquisition component 520 for acquiring a deduplication data fingerprint when performing read and write operations on the cache storage 550.

[0108] The data fingerprint acquisition component 520 includes a read operation component 521 for acquiring a data fingerprint of a read operation from the deduplication metadata 541 of the deduplication system 540, where the deduplication metadata 541 maps address aliases from different software logical layers or different client domains to deduplication data fingerprints, and a write operation component 522 for applying the deduplication fingerprinting function to the data of a write operation.

[0109] The cache storage system indexing system 510 includes an array lookup component 511 for looking up a deduplication-based address array 514 to obtain the cache address of data for data operations to the cache storage 550 using a reference based on the data fingerprint of the data.

[0110] The cache storage system indexing system 510 may include a hit matching component 512 for matching a data fingerprint with the data fingerprint of the retrieved data.

[0111] The cache storage system indexing system 510 may include a reference unification component 513 for discovering multiple references to a data source by using data fingerprint addressing, unifying the references to the data source, and thereby enhancing deduplication.

[0112] Figure 6 shows a block diagram of the components of a computing system used in a cache indexing system 110 according to an embodiment of the present invention. It should be understood that Figure 6 provides only an example of one implementation and does not imply any limitations regarding the environment in which different embodiments may be implemented. Many modifications may be made to the depicted environment.

[0113] The computing system may include one or more processors 602, one or more computer-readable RAMs 604, one or more computer-readable ROMs 606, one or more computer-readable storage media 608, device drivers 612, read / write drives or interfaces 614, and network adapters or interfaces 616, all interconnected on a communication fabric 618. The communication fabric 618 can be implemented in any architecture designed to pass data or control information, or both, between processors (such as microprocessors, communication and network processors), system memory, peripheral devices, and any other hardware components in the system.

[0114] One or more operating systems 610 and application programs 611 are stored in one or more computer-readable storage media 608 for execution by one or more processors 602 via one or more RAMs 604 (typically including cache memory). In the illustrated embodiment, each of the computer-readable storage media 608 may be a magnetic disk storage device of an internal hard drive, a CD-ROM, DVD, memory stick, magnetic tape, magnetic disk, optical disk, RAM, ROM, EPROM, flash memory or any other computer-readable storage medium capable of storing computer programs and digital information, according to embodiments of the present invention.

[0115] The computing system may also include an R / W drive or interface 614 for reading from and writing to one or more portable computer-readable storage media 626. An application program 611 on the computing system may be stored in one or more portable computer-readable storage media 626, read via their respective R / W drives or interfaces 614, and loaded into their respective computer-readable storage media 608.

[0116] The computing system may also include a network adapter or interface 616, such as a TCP / IP adapter card or a wireless communication adapter. Application programs 611 on the computing system can be downloaded to the computing device from an external computer or external storage device via a network (e.g., the Internet, a local area network, or other wide area network or wireless network) and the network adapter or interface 616. The program may be loaded from the network adapter or interface 616 onto a computer-readable storage medium 608. The network may include copper wire, optical fiber, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.

[0117] The computing system may also include a display screen 620, a keyboard or keypad 622, and a computer mouse or touchpad 624. The device driver 612 interfaces to the display screen 620 for imaging, the keyboard or keypad 622, the computer mouse or touchpad 624, or to the display screen 620 for alphanumeric character input and pressure sensing for user selection, or a combination thereof. The device driver 612, the R / W drive or interface 614, and the network adapter or interface 616 may include hardware and software stored in a computer-readable storage medium 608 or ROM 606 or both.

[0118] The present invention may be a system, method, or computer program product or combination thereof, integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium storing computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0119] A computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. A computer-readable storage medium may, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or a suitable combination thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, EPROM (or flash memory), SRAM, CD-ROM, DVD, memory stick, floppy disk, punch cards, or grooved raised structures, and mechanically encoded devices on which instructions are recorded, and suitable combinations thereof. The computer-readable storage medium as used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0120] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof). The network consists of copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. The network adapter card or network interface of each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on the computer-readable storage medium within each computing / processing device.

[0121] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++ and procedural programming languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions are executable as a standalone software package, either entirely on the user's computer or partially on the user's computer. Alternatively, they may be executable partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer-readable program instructions by personalizing them using state information of computer-readable program instructions in order to perform aspects of the present invention.

[0122] Aspects of the present invention are described herein with reference to flowcharts or block diagrams, or both, of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in a flowchart or block diagram, or both, and any combination of blocks in a flowchart or block diagram, or both, can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions can be provided to a computer processor or other programmable data processing device to generate a machine, such that instructions executed via the processor of the computer or other programmable data processing device generate means for implementing functions / operations specified in one or more blocks of a flowchart or block diagram or both. These computer-readable program instructions can also be stored in a computer-readable storage medium that can be connected to a computer, a programmable data processing device, or other device or combination of devices that function in a particular way, such that the computer-readable storage medium on which the instructions are stored constitutes one of the outputs containing instructions that implement the modes of functions / operations specified in one or more blocks of a flowchart or block diagram or both.

[0124] Computer-readable program instructions, like instructions that perform a function / action specified in one or more blocks of a flowchart or block diagram or both on a computer, other programmable device, or other device, can also be loaded into a computer, other programmable data processing device, or other device and perform a series of operational steps on the computer, other programmable device, or other device to produce a computer-implemented process.

[0125] The flowcharts and block diagrams in the figures illustrate the configuration, function, and operation of executable implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction, which constitutes one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions shown in the blocks may differ from the order shown in the figures. For example, two blocks shown consecutively may actually be achieved as a single step, executed simultaneously, substantially simultaneously, partially or entirely in overlapping time, or the blocks may be executed in reverse order depending on the functions involved. It should also be noted that each block in a block diagram or flowchart diagram, or both, and any combination of blocks in a block diagram or flowchart diagram, or both, can be implemented by a special-purpose hardware-based system that performs a specified function or operation, or a combination of special-purpose hardware and computer instructions.

[0126] <Cloud Computing> This disclosure includes a detailed description of cloud computing, but the implementations of the teachings described herein are not limited to cloud computing environments. Rather, embodiments of the present invention can be implemented in any other type of computer environment that is currently known or may be developed in the future.

[0127] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with service providers. This cloud model may include at least five characteristics, at least three service models, and at least four implementation models.

[0128] The characteristics are as follows:

[0129] On-demand self-service: Cloud consumers can unilaterally prepare computing power, such as server time and network storage, automatically as needed, without requiring human interaction with service providers.

[0130] Broad network access: Computing power is available over the network and accessible through standard mechanisms. This facilitates utilization by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, PDAs).

[0131] Resource pooling: A provider's computing resources are pooled and delivered to multiple consumers using a multi-tenant model. Various physical and virtual resources are dynamically allocated and reallocated as needed. Generally, consumers have a sense of location independence because they do not manage or know the exact location of the resources provided. However, consumers may be able to identify the location at a higher level of abstraction (e.g., country, state, data center).

[0132] Rapid Elasticity: Computing power can be prepared quickly and flexibly, allowing it to scale out automatically and immediately, and to be quickly released and scale in immediately. To consumers, the computing power available for preparation often appears unlimited and can be purchased in any quantity at any time.

[0133] Measured Services: Cloud systems leverage metric capabilities at a certain level of abstraction, appropriate for the type of service (e.g., storage, processing, bandwidth, active user accounts), to automatically control and optimize resource usage. Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0134] The service model is as follows:

[0135] Software as a Service (SaaS): The functionality offered to consumers is the ability to use the provider's applications running on a cloud infrastructure. These applications can be accessed from various client devices via thin client interfaces such as web browsers (e.g., webmail). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, except for configuring a limited number of user-specific applications.

[0136] Platform as a Service (PaaS): The functionality offered to consumers is the ability to deploy applications they have created or acquired to cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, and storage, but they can control the deployed applications and, in some cases, the configuration of their hosting environment.

[0137] Infrastructure as a Service (IaaS): The functionality provided to consumers is the provision of processors, storage, networking, and other basic computing resources that enable consumers to deploy and run any software, including operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they can control the operating system, storage, and deployed applications, and in some cases, partially control certain network components (e.g., host firewalls).

[0138] The deployment model is as follows:

[0139] Private Cloud: This cloud infrastructure is operated exclusively for a specific organization. This cloud infrastructure can be managed by that organization or a third party and can reside on-premises or off-premises.

[0140] Community Cloud: This cloud infrastructure is shared by multiple organizations to support a specific community with common interests (e.g., mission, security requirements, policies, and compliance). This cloud infrastructure can be managed by the organization or a third party and can reside on-premises or off-premises.

[0141] Public Cloud: This cloud infrastructure is provided to a large number of people or large industry groups and is owned by organizations that sell cloud services.

[0142] Hybrid Cloud: This cloud infrastructure combines two or more cloud models (private, community, or public). While maintaining the unique entities of each model, they are bound together by standards or individual technologies to achieve data and application portability (e.g., cloud bursting for load balancing across clouds).

[0143] Cloud computing environments are service-oriented environments that emphasize statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is the infrastructure, which includes a network of interconnected nodes.

[0144] Figure 7 depicts an exemplary cloud computing environment 50. As shown in the figure, the cloud computing environment 50 includes one or more cloud computing nodes 10. Local computer devices used by cloud consumers (e.g., PDAs or mobile phones 54A, desktop computers 54B, laptop computers 54C, or automotive computer systems 54N, or a combination thereof) can communicate with these nodes. The nodes 10 can communicate with each other. The nodes 10 can be grouped physically or virtually (not shown) in one or more networks, such as the private, community, public, or hybrid clouds or a combination thereof. This allows the cloud computing environment 50 to provide infrastructure, platforms, or software as a service, or a combination thereof, without requiring cloud consumers to maintain resources on their local computer devices. Note that the types of computer devices 54A-N shown in Figure 7 are merely examples, and it should be understood that the computing nodes 10 and the cloud computing environment 50 can communicate with any type of electronic device via any type of network or network addressable connection (e.g., using a web browser) or both.

[0145] Referring now to Figure 8, a set of functional abstraction layers provided by the cloud computing environment 50 (Figure 7) is shown. It should be understood that the components, layers, and functions shown in Figure 8 are merely illustrative, and the embodiments of the present invention are not limited to these. As illustrated, the following layers and corresponding functions are provided.

[0146] The hardware and software layer 60 includes hardware components and software components. Examples of hardware components include a mainframe 61, a reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage 65, and a network and network components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0147] The virtualization layer 70 provides an abstraction layer. From this layer, for example, the following virtual entities can be provided: virtual servers 71, virtual storage 72, virtual networks 73 including virtual private networks, virtual applications and operating systems 74, and virtual clients 75.

[0148] As an example, the management layer 80 can provide the following functions: Resource preparation 81 enables the dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 enables cost tracking as resources are used within the cloud computing environment and billing or invoicing for the consumption of these resources. As an example, these resources may include licenses for application software. Security enables not only protection of data and other resources but also identification and verification of cloud consumers and tasks. The user portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 enables the allocation and management of cloud computing resources to ensure that requested service levels are met. Service Level Agreement (SLA) planning and execution 85 enables the pre-arrangement and procurement of cloud computing resources that are expected to be needed in the future in accordance with the SLA.

[0149] Workload layer 90 provides examples of the capabilities available to the cloud computing environment. Examples of workloads and capabilities available from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom education delivery 93, data analytics processing 94, transaction processing 95, and cache indexing processing 96.

[0150] The computer program product of the present invention includes one or more computer-readable hardware storage devices that store computer-readable program code, the program code being executable by one or more processors to carry out the method of the present invention.

[0151] The computer system of the present invention comprises one or more processors, one or more memories, and one or more computer-readable hardware storage devices, wherein the one or more hardware storage devices contain program code that can be executed by one or more processors via one or more memories, and the method of the present invention is implemented.

[0152] The descriptions of various embodiments of the present invention are presented for illustrative purposes only and are not intended to be exhaustive, nor are they intended to limit the disclosed embodiments. It will be apparent to those skilled in the art that many modifications and changes are possible without departing from the scope of the embodiments described. The terminology used herein has been selected to best describe the principles of the embodiments, their practical application to market-based technologies, or technical improvements, or to enable those skilled in the art to understand the embodiments described herein.

[0153] Improvements and modifications can be made to the foregoing without departing from the scope of the present invention.

Claims

1. A computer implementation method for cache storage system indexing, wherein the computer implementation method is Indexing data addresses in a cache storage system based on a data fingerprint of cached data by one or more computer processors, wherein the data fingerprint is generated by a deduplication fingerprinting function used to refer to the deduplication of data in the cache storage system. Includes, A computer implementation method wherein indexing the data address is performed based only on a subset of the middle portion of the byte sequence of the data fingerprint as a reference to an array that stores cache addresses in the cache address space of the cache storage system.

2. The computer implementation method according to claim 1, wherein the data fingerprint is obtained by a read operation from deduplication metadata that maps different address aliases from different software logical layers to deduplication data fingerprints.

3. The computer implementation method according to claim 1, wherein the data fingerprint is obtained by a read operation from deduplication metadata that maps different address aliases from different client domains to deduplication data fingerprints.

4. The computer implementation method according to any one of claims 1 to 3, wherein the cache storage system is a single uniform cache for read / write operations in multiple layers of the storage subsystem, and the indexing provides uniform indexing across the multiple layers.

5. The computer implementation method according to any one of claims 1 to 3, wherein the cache storage system is a sharded cache with sharding based on the data fingerprint generated by the deduplication fingerprinting function.

6. A computer implementation method for data manipulation in a cache storage system, Obtaining a data fingerprint generated by using a deduplication fingerprinting function used for data deduplication in the cache storage system by one or more computer processors, by applying the deduplication fingerprinting function to the data of a write operation, thereby obtaining the data fingerprint for the data of the data operation, Using an indexing service to the cache storage system having an address schema based on the data fingerprint of the data of the write operation, Includes, A computer implementation method in which the indexing service uses a reference based only on a subset of the middle portion of the byte sequence of the data fingerprint as an address to an array that stores cache addresses in the cache address space of the cache storage system.

7. The aforementioned data manipulation is a write operation, and the computer implementation method is Receiving data write operations by one or more computer processors, One or more computer processors calculate a data fingerprint of the data of the data write operation using a fingerprint function used for data deduplication in the cache storage system, One or more computer processors create cache entries for the data of the data write operation, To address the cache entries in the cache storage system, the data fingerprint is used, The computer implementation method according to claim 6, further comprising:

8. The aforementioned data manipulation is a read operation, and the computer implementation method is One or more computer processors receive the data read operation having a logical address for the data of the data read operation, One or more computer processors obtain a data fingerprint of the data for the data reading operation from the deduplication metadata by referring to the logical address, The data fingerprint is used to address the data in the cache storage system from which the data to be read in the aforementioned data reading operation is to be read, The computer implementation method according to claim 6, further comprising:

9. The computer implementation method according to claim 8, wherein obtaining the data fingerprint is performed after a cache miss using the original addressing scheme of the cache storage system when the read operation comes down to the deduplication layer, since it could not be served using the logical address.

10. The computer implementation method according to claim 8, wherein obtaining the data fingerprint and using the data fingerprint to address the data are performed before the original addressing scheme.

11. One or more computer processors enable false hits of the cache storage system using the data fingerprint to address the data for the data read operation, One or more computer processors compare the data fingerprint of the retrieved data with the data fingerprint of the retrieved data, The computer implementation method according to claim 8, further comprising:

12. One or more computer processors use a subset of the data fingerprints as addresses to arrays that store cache addresses, One or more computer processors compare the data fingerprint of the retrieved data with the data fingerprint of the retrieved data, The computer implementation method according to claim 8, further comprising:

13. One or more computer processors discover multiple references to a data source by using addressing with the data fingerprint, One or more computer processors unify the references to the data source, thereby increasing the deduplication, The computer implementation method according to claim 8, further comprising:

14. The computer implementation method according to claim 8, wherein obtaining the data fingerprint and using the data fingerprint to address the data are performed in parallel with the original addressing scheme.

15. One or more computer processors, One or more computer-readable storage media, The program instructions include, which are stored in the one or more computer-readable storage media for execution by at least one of the one or more computer processors, A program instruction for creating an array that stores cache addresses in a shared cache address space, wherein the array has references based on data fingerprints of cached data, and the data fingerprints are generated by a deduplication fingerprinting function used for data deduplication in a cache storage system. The indexing of data addresses is performed based only on a subset of the middle portion of the byte sequence of the data fingerprint, as a reference to an array that stores cache addresses in the cache address space of the cache storage system, according to a program instruction. A computer system, including a computer system.

16. Stored in the one or more computer-readable storage media for execution by at least one of the one or more computer processors, When performing read and write operations on the aforementioned cache storage system, the method involves obtaining a deduplication data fingerprint, Obtaining a data fingerprint for a read operation from deduplication metadata, wherein the deduplication metadata maps address aliases from different software logical layers or different client domains to the deduplication data fingerprint, Applying a deduplication fingerprinting function to the data of the write operation, The computer system according to claim 15, further comprising one or more program instructions that perform the action of obtaining, including

17. The computer system includes a single uniform cache for read / write operations across multiple layers of the storage subsystem, and the computer system provides uniform indexing across the multiple layers. The computer system according to claim 15.

18. Stored in the one or more computer-readable storage media for execution by at least one of the one or more computer processors, To create a sharded cache with sharding based on the data fingerprint generated by the aforementioned deduplication fingerprinting function. The computer system according to claim 15, further comprising one or more program instructions for executing the

19. A system that performs indexing on a cache storage system, A data fingerprint acquisition component for acquiring a data fingerprint for data operations by applying the deduplication fingerprint function to the data of a write operation and accessing deduplication metadata for a read operation, in order to acquire a data fingerprint generated by using the deduplication fingerprint function used for deduplication of data in the cache storage system, wherein the indexing of data addresses is performed based only on a subset of the middle part of the byte sequence of the data fingerprint as a reference to an array that stores cache addresses in the cache address space of the cache storage system, An array lookup component for obtaining a cache address for the data of the data operation to the cache storage system by using a reference based on the data fingerprint of the data of the data operation, A system that performs indexing on a cache storage system, including [specific components / features].

20. Receiving a data write operation, Using the fingerprint function used for data deduplication in the cache storage system, calculate the data fingerprint of the data in the data write operation. Creating a cache entry for the data of the aforementioned data writing operation, To address the cache entries in the cache storage system, the data fingerprint is used, A write operation component for performing the following: Receiving the data read operation which has a logical address for the data of the data read operation, By referring to the aforementioned logical address, a data fingerprint of the data read from the data read operation is obtained from the deduplication metadata, The use of the data fingerprint to address the data in the data write operation and the data read operation within the cache storage system, A read operation component for performing the operation, A system for indexing to the cache storage system according to claim 19, further comprising:

21. A hit matching component for matching the data fingerprint of the retrieved data with the aforementioned data fingerprint. A system for indexing to the cache storage system according to claim 19, further comprising:

22. A reference unification component for discovering multiple references to a data source and unifying those references to the data source, thereby increasing the deduplication, by using addressing with the aforementioned data fingerprint. A system for indexing to the cache storage system according to claim 19, further comprising:

23. A sharded cache storage system using a data fingerprint address as a shard key, wherein the data fingerprint address is an address based on a data fingerprint obtained using a deduplication fingerprint function used for deduplication of data within the cache storage system, and the indexing of the data fingerprint address is performed based only on a subset of the middle portion of the byte sequence of the data fingerprint as a reference to an array that stores cache addresses in the cache address space of the cache storage system.

Images