Method and system for data deduplication and compression in untrusted storage systems

Abstract

A method for data deduplication and compression in untrusted storage systems is provided for more efficient storage of large amounts of data in a secure manner by maintaining data integrity. The data deduplication and compression in untrusted storage systems is performed with a system comprising a set of trusted servers that are configured to trust each other and share a common encryption key.

Description

Technical Field

[0001] This invention generally relates to the field of data protection and backup; more specifically, it relates to methods and systems for data deduplication and compression in untrusted storage systems. Background Technology

[0002] Typically, data backups are used to protect and recover data in the event of data loss in the primary storage system (such as a server). Examples of data loss events can include, but are not limited to, data corruption, hardware or software failures in the primary storage system, accidental data deletion, hacking, or malicious attacks. Therefore, for security reasons, separate backup systems or storage systems are widely used to store backups of data present in the primary storage system. However, over time, as data changes or any new data occupies a significant amount of storage space in a traditional storage system, the storage space becomes occupied. This is undesirable because it leads to a degraded storage system performance. Furthermore, the cost of data storage, along with all associated costs including storage hardware costs, remains a burden.

[0003] Typically, in many situations, such as in cloud environments, storage may be untrusted (i.e., untrusted storage). In such cases, data written to untrusted storage is encrypted. In other words, when using untrusted storage, there is a potential risk of data leakage if data is stored unencrypted. For example, any third party or untrusted entity could also read the stored data in untrusted storage (e.g., in a cloud environment). Therefore, encrypting data stored in such untrusted storage reduces the risk to data security. However, if the stored data is encrypted, it means that the data cannot be further compressed, and there are limitations on the encryption keys used, which in turn limits the amount of data reduction in such conventional storage systems (e.g., untrusted storage or backup systems). Furthermore, there is a problem with data deduplication in such traditional systems, because data deduplication can only be performed on encrypted data using the same encryption key. In addition, sometimes encrypted data needs to have its key updated, in which case data deduplication stops working, which adversely affects data protection, recovery, and data reduction.

[0004] Currently, many techniques are available for data reduction, such as fixed-size data deduplication. In fixed-size data deduplication, a given memory is divided into aligned blocks of fixed size, such as 8KB or 16KB. Furthermore, a strong hash signature is calculated for each block. If a block to be written to memory has the same hash signature as an already written block, the block is considered identical, and only a pointer (or reference) to that block is used. If the block is not identical, the data in that block is compressed before being written to memory. This traditional technique is inefficient and typically involves reducing the amount of data to be stored by eliminating redundant data. Therefore, the amount of data to be stored in traditional memory is not significantly reduced. Additionally, some storage systems allow backup storage to establish trust with a server (or client) by granting the backup storage access to the server's (or client's) encryption key. This allows the backup storage to decrypt, compress, and deduplicate data, and also re-encrypt data during reads. However, unless a trust relationship exists between the backup storage and the server writing to it, traditional techniques do not allow any compression of deduplicated data. Therefore, in traditional techniques, the deduplication and compression rates of encrypted data are much lower. Therefore, how to reduce data in untrusted storage systems remains a technical problem.

[0005] Therefore, based on the above discussion, it is necessary to overcome the aforementioned drawbacks associated with traditional data storage, which do not allow for any deduplication of data in the absence of mutual trust between the server and the storage. Summary of the Invention

[0006] This invention aims to provide a method and system for data deduplication and compression in untrusted storage systems. The invention seeks to provide a solution to the existing problem of inefficient data storage, specifically, how to further reduce data in untrusted storage compared to conventional methods and systems. The object of this invention is to provide a solution that at least partially addresses the problems encountered in the prior art and provides improved methods and systems for effective data deduplication and compression even in untrusted storage systems, thereby achieving efficient data storage (i.e., significantly reducing data stored in untrusted storage environments such as cloud environments).

[0007] The objective of this invention is achieved by the solution provided in the appended independent claims. Advantageous embodiments of the invention are further defined in the dependent claims.

[0008] In one aspect, the present invention provides a method for data deduplication and compression in an untrusted storage system, the method comprising: defining a first group of trusted servers, wherein each trusted server in the first group of trusted servers is used to use a first encryption key and a first hash function; separating the data in the trusted servers into aligned blocks of fixed size according to the data block size used for compression in the untrusted storage system; calculating a first hash value of the data; compressing the data of each block on the servers in the first group of trusted servers; encrypting the compressed data using the first encryption key; executing a write function via an application program interface (API) by sending the key ID of the first encryption key and the first hash value from the one or more trusted servers in the first group of trusted servers to the storage array of the untrusted storage system; and executing a read function via the API by decrypting the data of each block on the servers using a key matching the key ID of the first encryption key and decompressing the data.

[0009] The method described in this invention enables efficient compression and encryption of data stored in the untrusted storage from one or more servers in the first group of trusted servers. Furthermore, the key ID and hash value of the data (before compression and encryption) are stored together with the compressed and encrypted data in the untrusted storage, and the encryption key is stored together with the first group of trusted servers. Therefore, when new data is stored in the untrusted storage, the hash value of the new data is compared with the hash value of the already stored data to check whether the new data already exists in the untrusted storage. Thus, compared to conventional techniques that cannot deduplicate compressed and encrypted data in a mutually distrustful environment, data deduplication can be achieved between the first group of trusted servers and the untrusted storage without requiring mutual trust between them. Moreover, storing compressed and encrypted data in the untrusted storage allows for the secure storage of more data, which is significantly more cost-effective than conventional techniques. Therefore, the method described in this invention provides a new protocol that allows data compression on a host (e.g., one or more servers in the first group of trusted servers) while allowing the untrusted storage to deduplicate and store the compressed data, thereby allowing data reduction even when the storage is untrusted.

[0010] In one embodiment, executing the read function further includes verifying that the decompressed data has a correct hash value. By verifying that the decompressed data has a correct hash value, the integrity of the data can be maintained, thereby preserving accuracy during recovery when needed.

[0011] In another embodiment, executing the write function further includes using the hash value and key ID at the storage array to identify whether the same hash value already exists in the storage array for the first group of trusted servers. This invention achieves data deduplication by identifying whether the same hash value already exists in the storage array. Data deduplication is achieved by comparing the hash values ​​rather than directly comparing the data.

[0012] In another embodiment, executing the write function at the storage array further includes identifying whether the key ID of the first encryption key has been changed. By identifying whether the key ID of the first encryption key has been changed, the key ID stored in the untrusted storage can be updated based on changes to encryption keys stored in the first set of trusted servers.

[0013] In another embodiment, if the key ID of the encryption key remains unchanged, the method further includes the untrusted memory storing a pointer to the location of the data via the write function. By storing the pointer to the location of the data, new data identical to the already stored data will not be stored. In other words, if the key ID remains unchanged, the new data will only point to the old data. Therefore, compared with conventional techniques, the method described in this invention can improve storage capacity.

[0014] In another embodiment, if the key ID of the encryption key has been changed, the method further includes replacing the data encrypted with the first encryption key at the storage array with the data encrypted with the changed encryption key, and updating the key ID of the changed encryption key. By replacing the data encrypted with the first encryption key with the data encrypted with the changed encryption key, effective data deduplication and compression can be ensured in the untrusted storage system even if the key ID of the encryption key has been changed. This achieves efficient data storage (i.e., significantly reducing the amount of data stored in untrusted storage such as in a cloud environment). Therefore, in other words, even if the key ID of the data is replaced, compression and deduplication of the untrusted storage can be achieved using the API.

[0015] In another aspect, the present invention provides a system for data deduplication and compression in an untrusted storage system, the system comprising: an untrusted storage device including fixed-size aligned blocks; trusted servers using an encryption key, wherein the trusted servers are grouped into one or more groups of trusted servers based on the encryption key; and an application programming interface (API) for executing write and read functions between the trusted servers and the untrusted storage device, wherein one or more of the trusted servers in the group are used to perform the data deduplication and compression on the untrusted storage system.

[0016] The system described in this invention provides efficient data storage (i.e., significantly reducing the amount of data stored in untrusted storage environments such as cloud environments). The system allows for effective data deduplication and compression in the untrusted storage, thereby increasing the data storage capacity of the untrusted storage. Data deduplication and compression in the untrusted storage are achieved using an application program interface (API) for executing write and read functions between the trusted server and the untrusted storage.

[0017] In another aspect, the present invention provides a computer-readable medium including computer-readable instructions for implementing an application programming interface (API) for data deduplication and compression in the untrusted storage system according to the methods described in the foregoing aspects. Compared to conventional techniques that require mutual trust for data deduplication, the computer-readable medium of the present invention can perform data deduplication and compression on data stored in the untrusted storage system without requiring mutual trust between the trusted server (transmitting the data) and the untrusted storage system (stores the data backup).

[0018] It should be understood that all the above-described embodiments can be combined together. It should be noted that the devices, elements, circuits, units, and apparatuses described in this application can all be implemented by software or hardware elements or any combination thereof. All steps performed by the various entities described in this application and the functions to be performed by the various entities are intended to indicate that each entity is suitable for or used to perform its respective steps and functions. Although in the following description of specific embodiments, specific functions or steps performed by external entities are not reflected in the description of the specific elements of the entity performing the specific steps or functions, it should be apparent to those skilled in the art that these methods and functions can be implemented in their respective hardware or software elements or any combination thereof. It should be understood that the features of the invention can be readily combined in various combinations without departing from the scope of the invention as defined by the appended claims.

[0019] Other aspects, advantages, features, and objects of the invention will be clearly described by the accompanying drawings and the detailed description of illustrative embodiments interpreted in conjunction with the appended claims. Attached Figure Description

[0020] The following detailed description of the invention and illustrative embodiments can be better understood by reading in conjunction with the accompanying drawings. Exemplary structures of the invention are shown in the drawings for illustrative purposes. However, the invention is not limited to the specific methods and means disclosed herein. Furthermore, those skilled in the art will understand that these drawings are not drawn to scale. Where possible, the same elements are represented by the same numbers.

[0021] Embodiments of this disclosure are described herein by way of example only, in conjunction with the accompanying drawings.

[0022] Figure 1A A flowchart of a method for data deduplication and compression in an untrusted storage system provided by an embodiment of the present invention;

[0023] Figure 1B A network environment diagram of a system for data deduplication and compression in an untrusted storage system provided in an embodiment of the present invention;

[0024] Figure 2A Block diagrams of various exemplary components of an untrusted memory provided for embodiments of the present invention;

[0025] Figure 2B A block diagram of various exemplary components of a trusted server provided in embodiments of the present invention;

[0026] Figure 3 An exemplary implementation scenario of a system for data deduplication and compression in an untrusted storage system, provided by another embodiment of the present invention, is illustrated.

[0027] Figure 4 An untrusted memory for data deduplication and compression in an untrusted storage system is shown according to another embodiment of the present invention.

[0028] In the accompanying diagrams, underlined numbers indicate the item to which the underlined number is located or the item adjacent to the underlined number. Ununderlined numbers are associated with the item identified by the line linking the ununderlined number to that item. When a number is ununderlined but has an associated arrow, the ununderlined number identifies the general item pointed to by the arrow. Detailed Implementation

[0029] The following detailed description illustrates embodiments of the present invention and their implementation. While some embodiments of the present invention have been disclosed, those skilled in the art will recognize that other embodiments for carrying out or practicing the present invention can also be implemented.

[0030] Figure 1A A flowchart illustrating a method for data deduplication and compression in an untrusted storage system, provided as an embodiment of the present invention. In one aspect, the present invention provides a method 100A for data deduplication and compression in an untrusted storage system, the method 100A comprising:

[0031] Define a first group of trusted servers, wherein each trusted server in the first group of trusted servers is used with a first encryption key and a first hash function;

[0032] Based on the block size of the data used for compression in the untrusted storage system, the data in the trusted server is separated into aligned blocks of a fixed size;

[0033] Calculate the first hash value of the data;

[0034] Compress the data in each block on the server in the first group of trusted servers;

[0035] The compressed data is encrypted using the first encryption key;

[0036] The write function is executed via an application program interface (API) by sending the key ID of the first encryption key and the first hash value from one or more trusted servers in the first group of trusted servers to the storage array of the untrusted storage system.

[0037] The server decrypts and decompresses the data of each block using a key that matches the key ID of the first encryption key, and executes a read function via the API.

[0038] The method 100A includes steps S1, S2, S3, S4, S5, S6, and S7. Steps S1 to S7 can be performed in any reasonable order to achieve the objectives of the disclosed embodiments. Unless a particular method step is a necessary prerequisite for performing any other method steps, Figure 1A The description and accompanying notes do not necessarily imply a specific order of the disclosed steps of method 100A. Individual method steps may be executed simultaneously or nearly simultaneously, sequentially, or in parallel. Reference Figure 1A The figure also shows an untrusted storage system 102 and a trusted storage system 104 (e.g., Figure 1B(It is described in detail here). The untrusted storage system 102 includes a storage array 106 having one or more untrusted memories, such as untrusted memory 106A (for data backup). The trusted server 104 includes one or more host servers, such as servers 104A to 104N.

[0039] In step S1, method 100A includes defining a first group of trusted servers, wherein each trusted server in the first group of trusted servers is used with a first encryption key and a first hash function. Typically, the trusted server 104 refers to a server trusted in terms of data security. Specifically, the trusted server 104 is a server that can communicate with each other to share the same encryption key or has a key manager for sharing the same encryption key. The first group of trusted servers refers to at least one group of mutually trusted servers (e.g., servers 104A and 104B). The first group of trusted servers is defined so that deduplication is performed specifically for the first group of trusted servers (i.e., servers within a host group), so that data in one host group is not overwritten by another host group during backup. Furthermore, a host group can share the same encryption key (e.g., the first encryption key) and the same hash function (e.g., the first hash function). In one example, the first group of trusted servers includes a group of host servers of a given organization, a team or department within the organization, or any entity. In another example, the first group of trusted servers includes servers of different participating entities that trust each other and share data and store the data in the untrusted storage system 102. The first group of trusted servers can be defined by one of the trusted servers 104, which can act as a management server or be based on user input, such as user configuration or settings. Each trusted server is used to encrypt data using an encryption algorithm using the first encryption key and to hash the data using the first hash function.

[0040] According to one embodiment, method 100A may further include one or more other groups of trusted servers, such as a second group of trusted servers, a third group of trusted servers, etc., wherein each trusted server in a corresponding server group trusts only servers in that group or cluster. The first hash function is agreed upon by the first group of trusted servers, such that each group of trusted servers has a different hash function to avoid collisions. During hash processing and encryption, a given trusted server sends compressed encrypted data and the hash value of the data before encryption to the untrusted storage system 102.

[0041] According to one embodiment, the encryption keys used by the first group of trusted servers can be stored in the key manager. In one example, the key manager may include an index of the encryption keys used by the respective groups of the different trusted servers. Furthermore, the key manager may include a key generation algorithm to generate the encryption keys used by the first group of trusted servers. According to one embodiment, each trusted server in the first group of trusted servers can communicate with other trusted servers to share the encryption keys. If the same encryption key and the same hash function are used, no communication is required, and only a predefined protocol can be used.

[0042] In step S2, method 100A further includes splitting the data in the trusted server 104 into fixed-size aligned blocks based on the data block size used for compression in the untrusted storage system 102. In other words, when the trusted server 104 attempts to write data to the untrusted storage system 102, it splits (i.e., creates data blocks) the data into fixed-size aligned blocks based on the storage block size used for compression. The data is stored in the untrusted storage system 102 in the form of data blocks. Here, the block size refers to the amount of data that a block can store (e.g., in kilobytes). Splitting the data into fixed-size aligned blocks at a given trusted server (e.g., server 104A) increases storage capacity and allows for efficient data retrieval from the untrusted storage system 102. Advantageously, splitting the data into blocks also improves the data indexing in the untrusted storage system 102. As a result of splitting the data into blocks, when writing data to the untrusted storage system 102, the write size is aligned with the compressed block size of the untrusted storage system 102, so that the size of each write will be a multiple of the compressed block size of the untrusted storage system. In one example, data splitting can be based on data type (e.g., from the same file or file type) or metadata associated with the data. In another example, data splitting can be performed based on data creation time, user-defined data backup time, data importance, alphabetical order of data, etc.

[0043] The untrusted storage system 102 is a block-based memory. In one example, the basic block size is 8 kilobytes, meaning that all data inputs and outputs to each untrusted memory in the untrusted storage system 102 must be aligned to an 8-kilobyte offset and the size must be a multiple of 8 kilobytes (i.e., 24 kilobytes, 40 kilobytes, 128 kilobytes, etc.). The write size (before compression) must be a multiple of 8 kilobytes (i.e., depending on the storage block size), for example, writing 8 kilobytes, 16 kilobytes, 24 kilobytes, 64 kilobytes, but not 4 kilobytes. In one example, if 4 kilobytes of data are to be written to offset 18 kilobytes, 16 kilobytes must first be read from offset 16 kilobytes, and the 4 kilobytes from offset 18 kilobytes to 22 kilobytes must be replaced with the new data, and then the 16 kilobytes must be written to offset 16 kilobytes again.

[0044] In one example, the untrusted storage system 102 includes at least one untrusted memory, such as untrusted memory 106A, for storing data received from the first group of trusted servers. In another example, the untrusted storage system 102 includes the storage array 106, which includes one or more untrusted memories, such as untrusted memory 106A, for storing data received from the first group of trusted servers.

[0045] In step S3, method 100A calculates a first hash value for the data. The first hash value is calculated at one of the trusted servers in the first group of trusted servers using the first hash function. In one example, the first hash value is calculated for each data block on the trusted server. The hash value refers to a fixed-size value of data of arbitrary size generated using a hash function or hash algorithm. In other words, the hash value refers to a fixed-size value representing the original data of arbitrary size. In one example, the hash value may be a hexadecimal string representing several characters of data. Examples of hash functions include, but are not limited to, SHA-1, SHA-2, MD5, and CRC32. Advantageously, the hash value enables comparison between two data blocks without requiring an actual comparison between the data in the two data blocks. The first hash value refers to the hash value generated by the trusted server for a given data block. The first hash value also enables the trusted server to identify and prevent the storage of duplicate data blocks.

[0046] In step S4, method 100A further includes compressing the data of each block on the servers in the first set of trusted servers. The data of each block on the server (e.g., server 104A, which is one of the trusted servers 104) is compressed to reduce the storage space occupied when the data is stored in the untrusted storage system 102. Therefore, the untrusted storage system 102 can store more data compared to storing uncompressed data. In one example, the first set of trusted servers uses a compression algorithm to compress the data. Examples of compression algorithms include, but are not limited to, LZ77, DEFLATE, LZR, compression based on a Convolutional Neural Network (CNN), and compression based on a Generative Adversarial Network (GAN). According to one embodiment, each block of the data is compressed to a level that preserves the authenticity of the data during decompression.

[0047] In step S5, method 100A further includes encrypting the compressed data using the first encryption key. At least one trusted server in the first group of trusted servers (e.g., server 104A) encrypts the compressed data using an encryption algorithm. Encrypting the compressed data prevents any unauthorized user, server, or third-party entity (e.g., in a cloud environment) from unnecessarily accessing the compressed data. For example, the data may contain confidential information associated with the first group of trusted servers, etc. Encrypting the compressed data prevents unauthorized users from stealing such confidential information. The first encryption key is a unique encryption key specifically used for the corresponding data block. Therefore, data can only be accessed using the first encryption key. Examples of encryption algorithms include, but are not limited to, Twofish, MD5, Advanced Encryption Standard (AES), International Data Encryption Algorithm (IDEA), etc.

[0048] In step S6, method 100A further includes executing a write function via application program interface (API) 108 by sending the key ID of the first encryption key and the first hash value from one or more trusted servers in the first group of trusted servers to the storage array 106 of the untrusted storage system 102. The key ID (i.e., key identifier) ​​is a unique identifier for a given encryption key. In one example, the key ID is used to associate the first encryption key stored in the first group of trusted servers with compressed data stored in the untrusted storage system 102. The first hash value is the hash value of the data generated before encryption at one of the trusted servers in the first group of trusted servers. The storage array 106 of the untrusted storage system 102 is an array in which data is stored in encrypted form to reduce any potential data leakage. In one example, the untrusted storage system 102 may be a cloud storage. The group of trusted servers trust each other but do not trust the untrusted storage system 102. Therefore, the first group of servers stores the compressed and encrypted data together with the key ID of the encryption key in the untrusted storage system 102, and stores the encryption key together with the first group of trusted servers.

[0049] In one example, the write function is represented by the function (1) shown below.

[0050] Write(offset,size,vector(compressed_data,compressed_size,hash of databefore compression,keyID of encryption)) (1), where,

[0051] 'offset' refers to the offset within the untrusted storage system 102. The offset can be a multiple of the compressed block size. For example, if a trusted server (e.g., server 104A) compresses data into 8-kilobyte blocks, the write offset will be a multiple of 8 kilobytes.

[0052] 'size' refers to the number of blocks to be written (uncompressed). The number of blocks is also a multiple of the compressed block size, for example, a multiple of 8 kilobytes;

[0053] `vector(compressed_data, compressed_size, hash of data before compression, keyID of encryption))` refers to a vector containing compressed data elements, and the size of the vector is the number of compressed data elements. For example, if the compressed block is 8 kilobytes and the size is 64 kilobytes, then the number of elements in the vector is 8 (i.e., 64 / 8 = 8).

[0054] 'compressed_data' refers to the data in a single compressed block;

[0055] 'compressed_size' refers to the size of the compressed data in bytes.

[0056] 'hash of data before compression' refers to the hash value of the data before compression and encryption, such as SHA2 (the hash value can also be encrypted using the same key as the compressed data); 'keyID of encryption' refers to the identifier of the encryption key used to encrypt the data after compression.

[0057] In step S7, method 100A further includes the server decrypting and decompressing the data of each block via API 108 using a key (e.g., server 104A) that matches the key ID of the first encryption key. Since the data is compressed and encrypted using the first encryption key, decrypting the data during the execution of the read function requires matching the key ID of the first encryption key so that the data can be decrypted using the first encryption key. Therefore, without the correct first encryption key, no unauthorized server can access the data. Advantageously, this allows method 100A to ensure that data is not stolen while also ensuring that data is stored in compressed form without requiring a mutual trust relationship between the first group of servers and the untrusted storage system 102. The method 100A of this invention provides a novel protocol that allows compression of data on the first group of trusted servers while allowing the untrusted storage system 102 to deduplicate and store data in compressed form, thereby allowing data reduction even when the storage is untrusted.

[0058] According to one embodiment, executing the read function further includes verifying that the decompressed data has the correct hash value. When reading data after executing the read function, a server in the first set of trusted servers (e.g., server 104A) will use a key matching the key ID to decrypt each data block. The server will then decompress the data and verify that the decompressed data does indeed have the correct hash value. This ensures data integrity and helps maintain accuracy during recovery if needed.

[0059] In one example, the read function performed is represented by the following function (2).

[0060] vector(compressed_data,compressed_size,hash of data beforecompression,keyID of encryption)=Read(offset,size) (2)

[0061] The parameters 'compressed_data', 'compressed_size', 'hash of data before compression', and 'keyID of encryption' in the read function (2) are the same as those in the write function, except that the vector in this function is the output instead of the input.

[0062] According to one embodiment, executing the write function further includes using the hash value and key ID at the storage array 106 to identify whether the same hash value used for the first group of trusted servers already exists in the storage array 106. During the execution of the write function, the hash value and key ID received from one of the first group of trusted servers are compared with the hash value and key ID already stored in the storage array 106 of the untrusted storage system 102. If, based on the comparison, the hash value and key ID already exist in the storage array 106 of the untrusted storage system 102, the data is no longer stored in the untrusted storage system 102. Therefore, by performing data deduplication using the method 100A of the present invention, the same (or similar) data will not be stored again in the untrusted storage system 102 based on the hash value and key ID of the data stored in the storage array 106. If, based on the comparison, the hash value and key ID do not exist in the storage array, the data will be stored in the untrusted storage system 102. In other words, if the hash value does not exist, the untrusted storage system 102 will create a new entry and store the compressed encrypted data with the hash value as an index for accessing the data. When the write function arrives from the first group of trusted servers, it performs an identification in the storage array 106 of the untrusted storage system 102 to determine whether the hash value and key ID received via the write function already exist in the storage array 106. In one example, the SHA-2 hash value of the data already stored in the untrusted storage system 102 is compared with the SHA-2 hash value of the data received from one of the first group of trusted servers.

[0063] Advantageously, compared to conventional techniques that require mutual trust between the server and the storage system to perform data deduplication, method 100A can perform data deduplication on data stored as compressed data without requiring mutual trust between the first group of servers and the untrusted storage system 102.

[0064] According to one embodiment, executing the write function at the storage array 106 further includes identifying whether the key ID of the first encryption key has been changed. After determining that the first hash value received by the write function already exists in the untrusted storage system 102, it is checked whether the key ID of the first encryption key has changed. If the hash value does not exist, the untrusted storage system 102 creates a new entry and stores the encrypted data with the hash value and compressed as an index for accessing the data. The key ID of the first encryption key may be changed periodically to improve security associated with encryption keys stored in the first set of trusted servers. In one example, the key ID may be changed by the server, or typically by the key manager notifying the server of the change. In one example, a user or the key manager may notify the server of the key ID change.

[0065] According to one embodiment, if the key ID of the encryption key has not changed, method 100A further includes storing a pointer to the data location by the untrusted memory via the write function. In other words, if the first hash value already exists in the storage array, the untrusted storage system 102 will check whether the key ID of the encryption key has changed compared to the last time. If the key ID of the encryption key has not changed (i.e., the encryption key has not changed), the new data (i.e., data received from the first set of trusted servers) will only point to the old data (i.e., data already stored in the untrusted storage system 102) via the pointer. In one example, the memory will only store a pointer to the data location from the new offset.

[0066] According to one embodiment, if the key ID of the encryption key has been changed, method 100A further includes replacing the data encrypted with the first encryption key at the storage array with the data encrypted with the changed encryption key, and updating the key ID of the changed encryption key. In other words, if the first hash value exists but the key ID has been updated, the untrusted storage system 102 will update the data and the encryption key, and all previous entries (i.e., stored data with the first hash value) will point to the data encrypted with the new encryption key. The pointer to this data is based on the hash value and therefore remains unchanged. In one example, if the key ID has been changed, the untrusted storage system 106A can also change the previously stored data and key ID, and store the new value instead of the last value. Therefore, this allows the storage key to be changed by method 100A while still preserving data deduplication, and only the new key ID needs to be used to write the data. This key ID change process signals all servers in the first group of servers to begin using the new key ID. After all storage systems have the new key ID, a background process can begin re-encrypting all data using the new key. In one example, the server may use the old key ID when re-encrypting the data.

[0067] In one exemplary embodiment, the first set of trusted servers sends compressed encrypted data along with the hash value of the unencrypted data to the untrusted storage system 102. Subsequent steps occur on the untrusted storage system 102 side. The untrusted storage system checks if the hash value already exists. If the hash value does not exist in the storage array, the untrusted storage system 102 creates a new entry and stores the compressed encrypted data with the hash value as an index for accessing the data. Additionally, the key ID of the encrypted data is stored. If the hash value exists in the storage array 106, the untrusted storage system 102 checks if the key ID of the encryption key has changed compared to the last time. If the key ID has not changed, the new data will only point to the old data already stored in the untrusted storage system. If the hash value exists but the key ID has been updated, the untrusted storage system updates the data and the encryption key, and all previous entries (i.e., stored data) will point to the data encrypted using the new encryption key.

[0068] Steps S1 to S7 are for illustrative purposes only. Other alternatives, such as adding one or more steps, removing one or more steps, or providing one or more steps in a different order, may be provided without departing from the scope of the claims herein.

[0069] In another aspect, the present invention provides a computer-readable medium including computer-readable instructions for implementing an application programming interface (API) for data deduplication and compression in the untrusted storage system according to method 100A. The computer-readable storage medium for providing non-transient memory may include, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. In yet another aspect, the present invention provides a computer program adapted to be executed by a device (e.g., one of the trusted servers 104 or the untrusted storage 106A) of method 100A. In yet another aspect, the present invention provides a computer program product comprising a non-transient computer-readable storage medium having processor-executable computer-readable instructions to perform method 100A. Examples of implementations of non-transient computer-readable storage media include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, secure digital (SD) card, solid-state drive (SSD), computer-readable storage media, or CPU cache memory.

[0070] Figure 1B This is a network environment diagram of a system for data deduplication and compression in an untrusted storage system, provided as an embodiment of the present invention. (Refer to...) Figure 1B The figure shows system 100B. System 100B includes the untrusted memory 106A, the trusted server 104, and the application programming interface 108. The trusted server 104 can be communicatively coupled to the untrusted memory 106A via a communication network 110. The trusted server 104 includes one or more groups of trusted servers, such as a first group of trusted servers (e.g., servers 104A and 104B) and a second group of trusted servers (e.g., servers 104C to 104N). The untrusted memory 106A includes a fixed-size alignment block 112. In one embodiment, the untrusted memory 106A is one of the untrusted memories of the storage array 106 of the untrusted storage system 102.

[0071] In another aspect, the present invention provides a system 100B for data deduplication and compression in an untrusted storage system, the system 100B comprising:

[0072] The untrusted memory 106A includes a fixed-size alignment block 112;

[0073] Trusted server 104 using encryption key, wherein the trusted server 104 is divided into one or more groups of trusted servers based on the encryption key;

[0074] An application program interface (API) 108 is used to execute write and read functions between the trusted server 104 and the untrusted storage system 106A, wherein one or more trusted servers in the trusted server 104 are used to perform data deduplication and compression on the untrusted storage system 102.

[0075] The untrusted memory 106A is one of the untrusted memories in the storage array 106 of the untrusted storage system 102. The untrusted memory 106A includes appropriate logic, circuitry, interfaces, or code for storing data (e.g., data associated with various software programs executing on the trusted server 104) in the fixed-size alignment block 112. In other words, the untrusted memory 106A is a backup memory in which the trusted server 104 can store data in the fixed-size alignment block 112 for any defined duration. In one example, the untrusted memory 106A is a block device or an electronic device that uses block devices for data storage. In one example, the untrusted memory 106A is a non-volatile mass storage device, such as a cloud-based storage medium. It should be understood that multiple untrusted memories of the untrusted storage system 102, such as storage array 106, can be used.

[0076] Each of the group or groups of trusted servers includes appropriate logic, circuitry, interfaces, or code for storing, processing, or transmitting information to the untrusted memory 106A via the communication network 110. Examples of the servers include, but are not limited to, storage servers, cloud servers, network servers, application servers, or combinations thereof. According to one embodiment, the server includes means of a physical or virtual computing entity capable of enhancing information to perform various computing tasks. In one example, the server may be a single hardware server. In another example, the server may be multiple hardware servers operating in a parallel or distributed architecture. In one example, the server may include components such as memory, processors, and network adapters to store, process, or share information with other computing components such as user devices. In one instance, the server is implemented as a computer program that provides various services (e.g., database services) to other devices, modules, or apparatuses.

[0077] The application programming interface 108 refers to a set of functions that can be called from an application to access features of another program, such as read or write functions, to execute between the untrusted memory 106A and the trusted server 104. In one embodiment, the application programming interface 108 is implemented in each trusted server 104. In another embodiment, the application programming interface 108 may be implemented in the untrusted memory 106A (or each trusted memory of the untrusted storage system 102). In one example, the application programming interface 108 is a software interface that includes one or more routines, data structures, object classes, and protocols supporting interaction between the trusted server 104 and one or more untrusted memories (e.g., the untrusted memory 106A).

[0078] The communication network 110 includes a medium (e.g., a communication channel) through which the trusted server 104 communicates with the untrusted memory 106A. The communication network 110 can be a wired or wireless communication network. Examples of the communication network 110 may include, but are not limited to, Wireless Fidelity (Wi-Fi) networks, Local Area Networks (LANs), Wireless Personal Area Networks (WPANs), Wireless Wide Area Networks (WWANs), Wireless Wide Area Networks (WWANs), cloud networks, Long Term Evolution (LTE) networks, Metropolitan Area Networks (MANs), or the Internet. The untrusted memory 106A and the trusted server 104 may be used to connect to the communication network 110 according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, infrared (IR), IEEE 802.11, 802.16, Long Term Evolution (LTE), Light Fidelity (Li-Fi), or other cellular communication protocols or Bluetooth (BT) communication protocols (including their variants).

[0079] The system 100B includes the untrusted memory 106A, which comprises fixed-size aligned blocks 112. The data is stored in the untrusted memory 106A in the form of data blocks. Here, "fixed size" refers to the amount of data that a block can store (e.g., in units of a defined block size). The trusted server 104 writes the data into the untrusted memory 106A by splitting (i.e., creating data blocks) the data into fixed-size aligned blocks according to the storage block size used for compression. The data is split into fixed-size aligned blocks at a given trusted server (e.g., server 104A) to increase storage capacity and efficiently retrieve data from the untrusted memory 106A.

[0080] The system 100B also includes the trusted server 104. The trusted server 104 uses the encryption key. The group or more trusted servers are grouped according to the encryption key. Each group of trusted servers is used to encrypt data using the encryption key and an encryption algorithm, which is used to store the data in encrypted form in the untrusted storage 106A. The encryption key may be stored in a key manager associated with each server in a given group of trusted servers. The trusted server 104 also uses a hash function to hash the data to be stored in the untrusted storage 106A. For example, the first group of trusted servers (i.e., servers 104A and 104B) may be used using the first encryption key and the first hash function.

[0081] The system 100B also includes an application programming interface 108 for executing write and read functions between the trusted server 104 and the untrusted storage 106A, wherein one or more trusted servers in the trusted server 104 are used to perform data deduplication and compression on the untrusted storage system 102. The write function is executed to store compressed encrypted data in the untrusted storage 106A, such that the compressed encrypted data can only be accessed by a given set of trusted servers (e.g., trusted servers sharing the same encryption key). The read function is executed to retrieve the stored compressed data from the untrusted storage 106A.

[0082] According to one embodiment, to perform data deduplication and compression, firstly, one or more trusted servers in the trusted servers 104 are used to define one or more groups of trusted servers, such as a first group of trusted servers (e.g., servers 104A and 104B) and a second group of trusted servers (e.g., servers 104C to 104N). Different groups of trusted servers are defined so that data deduplication can be performed on trusted servers within a host group, thus preventing data in one host group from being overwritten by another host group during backup. Then, one or more trusted servers in at least one group of trusted servers (e.g., the first group of trusted servers (e.g., servers 104A and 104B)) are also used to split the data in the trusted servers 104 into aligned blocks of fixed size according to the data block size used for compression in the untrusted storage system. Splitting the data into aligned blocks of fixed size improves storage capacity and allows for efficient data retrieval from the untrusted storage system 102. Advantageously, splitting the data into blocks also improves data indexing in the untrusted storage 106A (and the entire untrusted storage system 102).

[0083] According to one embodiment, one or more trusted servers in the trusted servers 104 (e.g., one or more trusted servers in the first group of trusted servers) are further used to calculate the hash value of the data. The hash value of the data is calculated using the hash function. The hash value enables comparison between two data blocks without requiring an actual comparison between the data in the two data blocks.

[0084] According to one embodiment, one or more trusted servers in the trusted server 104 (e.g., one or more trusted servers in the first group of trusted servers) are also used to compress the data in each block. Compressing the data in each block reduces the storage space occupied when the data is stored in the untrusted memory 106A. Therefore, the untrusted memory 106A can store more data compared to storing uncompressed data.

[0085] According to one embodiment, one or more trusted servers in the trusted server 104 (e.g., one or more trusted servers in the first group of trusted servers) are further configured to encrypt the compressed data using the first encryption key. Encrypting the compressed data prevents any unauthorized user, server, or third-party entity from unnecessarily accessing the compressed data.

[0086] According to one embodiment, the application programming interface 108 is further configured to use the hash value and key ID at the untrusted memory 106A to identify whether the same hash value already exists in the untrusted memory 106A of a given set of trusted servers (e.g., the first set of trusted servers). During the execution of the write function, the hash value and key ID received from the trusted server 104 are compared with the hash value and key ID already stored in the untrusted memory 106A.

[0087] According to one embodiment, the application programming interface 108 is further configured to identify whether the write function has been modified in order to execute the write function at the untrusted memory 106A. After determining that the first hash value received by the write function already exists in the untrusted memory 106A, it is checked whether the key ID of the first encryption key has changed. If the hash value does not exist, the untrusted memory 106A creates an entry and stores the encrypted data with the hash value, compressed as an index for accessing the data.

[0088] According to one embodiment, if the key ID of the encryption key has not changed, the system 100B is further configured to store a pointer to the data location by the untrusted memory 106A via the write function. In other words, if the hash value already exists in the storage array, the untrusted storage system 102 will check whether the key ID of the encryption key has changed compared to the last time. If the key ID of the encryption key has not changed, the new data (i.e., data received from the first group of trusted servers) will only point to the old data (i.e., data already stored in the untrusted storage system 102) via the pointer.

[0089] According to one embodiment, if the key ID of the encryption key has been changed, the system is further configured to replace the data encrypted with the encryption key with the data encrypted with the changed encryption key at the untrusted memory 106A, and update the data with the key ID of the changed encryption key. In other words, if the hash value exists but the key ID has been updated, the untrusted memory 106A is used to update the data and the encryption key, and all previous entries will point to the data encrypted with the new encryption key. The pointer to this data is based on the hash value and therefore remains unchanged.

[0090] According to one embodiment, if the write operation is not aligned with the block size, a read function is executed before the write operation so that the data is available for writing. In other words, the data write operation is performed for a specified size (e.g., 8 kilobytes), and if the write operation is not aligned with the block size, a read function is executed first.

[0091] According to one embodiment, for example, during recovery, the application programming interface 108 is used (by one or more trusted servers in the trusted servers 104) to decrypt and decompress the data of each block using a key that matches the key ID of the encryption key. Since the data is compressed and encrypted using the encryption key, decrypting the data during the execution of the read function requires matching the key ID of the encryption key in order to be able to decrypt the data using the encryption key. Therefore, without the correct encryption key, no unauthorized server can access the data.

[0092] According to one embodiment, the application programming interface 108 is also used to verify that the decompressed data has the correct hash value. When reading data after executing the read function, the one or more trusted servers (e.g., server 104A) will decrypt each data block using a key matching the key ID through the application programming interface 108. Then, the one or more trusted servers (e.g., server 104A) will decompress the data and verify that the decompressed data does indeed have the correct hash value. This ensures data integrity and helps maintain accuracy during recovery if needed.

[0093] Figure 2A Block diagrams of various exemplary components of an untrusted memory provided for embodiments of the present invention. (See reference...) Figure 2A The figure shows the untrusted memory 106A. The untrusted memory 106A includes a first processor 202, a first transceiver 204, and a first memory 206. The first processor 202 can be communicatively coupled to the first transceiver 204 and the first memory 206. The untrusted memory 106A is coupled to the trusted server 104 through the communication network 110.

[0094] The first processor 202 is used to receive data from the trusted server 104. In one embodiment, the first processor 202 is used to execute instructions stored in the first memory 206. In one example, the first processor 202 may be a general-purpose processor. Other examples of the first processor 202 may include, but are not limited to, microprocessors, microcontrollers, complex instruction set computing (CISC) processors, application-specific integrated circuit (ASIC) processors, reduced instruction set computing (RISC) processors, very long instruction word (VLIW) processors, central processing units (CPUs), state machines, data processing units, and other processors or control circuits. Furthermore, the first processor 202 may refer to one or more separate processors, processing devices, or processing units as part of a machine, such as the untrusted memory 106A.

[0095] The first transceiver 204 includes appropriate logic, circuitry, and interfaces that can be used to communicate with one or more external devices (such as the trusted server 104). Examples of the first transceiver 204 may include, but are not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, or a subscriber identity module (SIM) card.

[0096] The first memory 206 refers to the main storage system of the untrusted memory 106A. The first memory 206 includes appropriate logic, circuitry, and interfaces for storing the fixed-size alignment block 112 and instructions executable by the first processor 202. Examples of implementations of the first memory 206 may include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, solid-state drive (SSD), or CPU cache memory. The first memory 206 may store an operating system or other program products (including one or more operating algorithms) to operate the untrusted memory 106A.

[0097] In operation, the first processor 202 of the untrusted memory 106A is used to receive and store the encrypted and compressed data, the key ID of the encryption key for the uncompressed data, and the hash value of the data in the first memory 206 (received from the trusted server 104 when the trusted server 104 executes a write function). The first processor 202 is also used to provide the compressed data from the first memory 206 to the trusted server 104 when a read function is executed.

[0098] Figure 2B A block diagram illustrating various exemplary components of a trusted server provided in embodiments of the present invention. (See reference...) Figure 2B The figure shows a trusted server 104A within the trusted server 104. The trusted server 104A further includes a second processor 208, a second transceiver 210, and a second memory 212. The second processor 208 is communicatively coupled to the second transceiver 210 and the second memory 212. The trusted server 104A is communicatively coupled to the untrusted memory 106A via the communication network 110.

[0099] The second processor 208 is used to execute instructions stored in the second memory 212. In one embodiment, the second processor 208 is used to execute instructions stored in the second memory 212. In one example, the second processor 208 may be a general-purpose processor. Other examples of the second processor 208 may include, but are not limited to, microprocessors, microcontrollers, complex instruction set computing (CISC) processors, application-specific integrated circuit (ASIC) processors, reduced instruction set computing (RISC) processors, very long instruction word (VLIW) processors, central processing units (CPUs), state machines, data processing units, and other processors or control circuits. Furthermore, the second processor 208 may refer to one or more separate processors, processing devices, or processing units as part of a machine, such as the trusted server 104A.

[0100] The second transceiver 210 includes appropriate logic, circuitry, and interfaces that can be used to communicate with one or more external devices (such as the untrusted storage system 102). Examples of the second transceiver 210 may include, but are not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, or a subscriber identity module (SIM) card.

[0101] The second memory 212 refers to the main storage system of the trusted server 104A. Examples of implementations of the second memory 212 may include, but are not limited to, electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), read-only memory (ROM), hard disk drive (HDD), flash memory, solid-state drive (SSD), or CPU cache memory. The second memory 212 may store an operating system or other program products (including one or more operating algorithms) to operate the trusted server 104A.

[0102] In operation, the second processor 208 of the trusted server 104A is used to define one or more groups of trusted servers. The second processor 208 is also used to separate data in the untrusted memory 106A into aligned blocks of fixed size according to the data block size used for compression in the untrusted memory 106A. The second processor 208 is also used to calculate the hash value of the data. The second processor 208 is also used to compress the data in each block. The second processor 208 is also used to encrypt the compressed data using the first encryption key. The second processor 208 is also used to enable a write function to be executed via the application programming interface 108 by sending the key ID of the encryption key and the hash value to the untrusted memory 106A (or the storage array 106). The second processor 208 is also used to enable a read function to be executed via the application programming interface 108 by decrypting the data in each block using a key matching the key ID of the encryption key and decompressing the data.

[0103] Figure 3 An exemplary implementation scenario of a system for data deduplication and compression in an untrusted storage system, according to another embodiment of the present invention, is shown. (Reference) Figure 3 The figure shows system 300. System 300 includes an untrusted storage 302, a first group of trusted servers 304, and a second group of trusted servers 306. The first group of trusted servers 304 includes trusted servers 308A, 308B, and 308C. The second group of trusted servers 306 includes servers 310A, 310B, and 310C.

[0104] The first group of trusted servers 304 can also be referred to as the first server area, and the second group of trusted servers 306 can also be referred to as the second server area. Servers in each group of trusted servers trust each other and use the same encryption key and the same hash function. Therefore, servers in each group of trusted servers can perform data deduplication among themselves.

[0105] The untrusted storage 302 includes: a first data deduplication area 312A for storing data received from the first group of trusted servers 304; and a second data deduplication area 312B for storing data received from the second group of trusted servers 306.

[0106] Data deduplication applies only to servers within each group of trusted servers, ensuring that data from one group of trusted servers is not overwritten by data from another group. This is guaranteed by the fact that each group of trusted servers has a different hash function (i.e., the hash function can be a cryptographic key specific to an unchanged group of servers).

[0107] The first group of trusted servers 304 and the second group of trusted servers 306 do not trust each other, therefore data deduplication is not performed between them. The first group of trusted servers 304 and the second group of trusted servers 306 also cannot read each other's data because they do not share the encryption key.

[0108] Each of the first group of trusted servers 304 and the second group of trusted servers 306 executes the write function to store the compressed and encrypted data along with its respective key ID and hash value in the untrusted storage 302. Furthermore, each of the first group of trusted servers 304 and the second group of trusted servers 306 is used to retrieve the compressed and encrypted data by executing the read function.

[0109] Advantageously, compared to conventional techniques that require mutual trust between the server and the storage system to perform data deduplication, the system 300 can perform data deduplication on data stored as compressed data without requiring mutual trust between the first set of trusted servers 304 and the untrusted storage 302.

[0110] Figure 4 An untrusted memory for data deduplication and compression in an untrusted storage system is illustrated according to another embodiment of the present invention. Reference Figure 4 The figure shows an untrusted memory 400. The untrusted memory 400 includes blocks 402A, 402B, 402C, and 402D for storing data.

[0111] Each block 402A, 402B, 402C, and 402D is 8 kilobytes in size. Data provided as input or received as output from the untrusted memory 400 is aligned to an 8 kilobyte offset and is a multiple of 8 kilobytes in size, such as 24 kilobytes, 40 kilobytes, 128 kilobytes, etc. In other words, the size written must be a multiple of 8 kilobytes, for example, 8 kilobytes, 16 kilobytes, 24 kilobytes, or 64 kilobytes. If 4 kilobytes of data 404 are to be written to offset 18 kilobytes, 16 kilobytes are first read from offset 16 kilobytes. Furthermore, the 4 kilobytes from offset 18 kilobytes to 22 kilobytes are replaced with the data 404, and then 16 kilobytes are written to offset 16 kilobytes.

[0112] The trusted server 104 writes the data 404 into the untrusted storage system by splitting the data 404 into fixed-size aligned blocks according to the storage block size used for compression. Splitting the data 404 into fixed-size aligned blocks at a given trusted server increases storage capacity and allows for efficient data retrieval from the untrusted storage system. Advantageously, splitting the data 404 into blocks also improves the indexing of the data 404 in the untrusted storage system.

[0113] Modifications to the embodiments of the invention described above may be made without departing from the scope of the invention as defined by the appended claims. Expressions such as “comprising,” “including,” “combined,” “having,” and “are” used to describe and claim the invention should be interpreted in a non-unique manner, allowing for the appearance of items, parts, or elements not explicitly described. Singular references may also be interpreted in relation to plurals, and vice versa. The term “exemplary” as used herein means “as an example, illustration, or description.” Any “exemplary” embodiment is not necessarily to be construed as preferential or superior to other embodiments, and / or does not exclude combinations of features from other embodiments. The term “optionally” as used herein means “provided in some embodiments and not in others.” It should be understood that, for clarity, certain features of this disclosure described in the context of a single embodiment may also be provided in combination in a single embodiment. Conversely, for brevity, various features of this disclosure described in the context of a single embodiment may also be provided individually or in any suitable combination or as any other suitable embodiment of this disclosure.

Claims

1. A method (100A) for data deduplication and compression in an untrusted storage system (102), characterized in that, The method includes: Define a first group of trusted servers, wherein each trusted server in the first group of trusted servers is used with a first encryption key and a first hash function; Based on the block size of the data used for compression in the untrusted storage system (102), the data in the trusted server (104) is separated into fixed-size aligned blocks that are aligned with the block size; Calculate the first hash value of the data; Compress the data in each block on the server in the first group of trusted servers; The compressed data is encrypted using the first encryption key; The write function is executed via application program interface (API) (108) by sending the key ID of the first encryption key and the first hash value from one or more trusted servers in the first group of trusted servers to the storage array (106) of the untrusted storage system (102). The server decrypts and decompresses the data of each block using a key that matches the key ID of the first encryption key, and executes the read function via the API (108).

2. The method (100A) according to claim 1, characterized in that, Executing the read function also includes verifying that the decompressed data has the correct hash value.

3. The method (100A) according to claim 1 or 2, characterized in that, Executing the write function also includes using the hash value at the storage array (106) and the key ID to identify whether the same hash value for the first group of trusted servers already exists in the storage array (106).

4. The method (100A) according to any one of claims 1 to 3, characterized in that, Executing the write function at the storage array (106) also includes identifying whether the key ID of the first encryption key has been changed.

5. The method (100A) according to claim 4, characterized in that, If the key ID of the encryption key is not changed, the method (100A) further includes the untrusted memory (106A, 302, 400) storing a pointer to the location of the data through the write function.

6. The method (100A) according to claim 4, characterized in that, If the key ID of the encryption key has been changed, the method (100A) further includes replacing the data encrypted with the first encryption key at the storage array (106) with the data encrypted with the changed encryption key, and updating the key ID of the changed encryption key.

7. A system (100B, 300) for data deduplication and compression in an untrusted storage system (102), characterized in that, The system (100B, 300) includes: Untrusted memory (106A, 302, 400) includes a fixed-size alignment block (112); Trusted servers (104) using encryption keys, wherein the trusted servers (104) are grouped into one or more groups of trusted servers (104A to 104N) based on the encryption keys; An application programming interface (108) for performing write and read functions between the trusted server (104) and the untrusted storage (106A, 302, 400), wherein one or more of the trusted servers in the group of trusted servers are used to perform data deduplication and compression as described in any one of claims 1 to 6 on the untrusted storage system (102).

8. A computer-readable medium comprising computer-readable instructions, characterized in that, The computer-readable instructions are used to implement the application programming interface (108) for data deduplication and compression in the untrusted storage system (102) according to the method (100A) of any one of claims 1 to 6.

Images