A two-way delta compression method for trusted execution environment

Abstract

The application discloses a bidirectional incremental compression method for a trusted execution environment and belongs to the technical field of safe file storage. The shielded memory space provided by the trusted execution environment is limited in size, and loading the required basic data block of the incremental compression generates expensive environment switching overhead, which causes the incremental compression to be unable to efficiently run in the trusted execution environment. n stored plaintext data blocks M of a batch B are simultaneously processed, q unique data blocks remaining after the data deduplication are inquired about whether similar basic data blocks M' exist in the system at one time, and the physical distribution of the basic blocks is judged. If the intermediate parameter x calculated based on the physical distribution is less than a preset threshold t, the stored plaintext block M is incrementally compressed based on the existing basic block, otherwise, the existing basic block M' is incrementally compressed based on the stored plaintext block M.

Description

Technical Field

[0001] This invention relates to the field of secure file storage technology, and in particular to a bidirectional incremental compression method for trusted execution environments. Background Technology

[0002] Incremental compression technology identifies commonalities between two similar data segments (called data blocks), retaining only the complete data block (called the base block) while partially preserving the unique portion of the other data block (called the incremental block). This aims to save transmission bandwidth or physical storage space. To incrementally compress a target data block, the system typically loads the corresponding base block into memory and then encodes the target block based on the base block.

[0003] File systems that support incremental compression typically use it as a supplementary technique to deduplication and lossless compression to further compress data. The data compression process in such systems mainly includes the following four steps: Step 1: Divide the file to be stored into several data blocks, and eliminate redundant information at the data block level by replacing the same redundant data blocks with pointers to the stored data (i.e., data deduplication). Step 2: For the remaining unique data block, further identify similar but different data blocks, and incrementally compress the data block to be stored using the existing base block; Step 3: For a unique data block that does not have a base block, use it as the base block and perform intra-block compression using lossless compression techniques; Step 4: Store the compressed base block and incremental block in a fixed-size data structure (called a container) for system access.

[0004] A Trusted Execution Environment (TEE) is a hardware and software combination technology that ensures the security of applications and data. It is a secure software environment that protects the programs and data running within it from malicious manipulation or tampering, ensuring the secure operation of mobile applications and services. Existing TEE technologies (such as Intel SGX and ARM TrustZone) provide encrypted and integrity-protected memory regions to create a shielded execution environment, combined with hardware protection to ensure the confidentiality and integrity of data within the shielded execution environment.

[0005] Existing technology (see Yang Z, Li J, Lee PP C. Secure and Lightweight Deduplicated Storage via Shielded Deduplication-Before-Encryption. USENIX Annual Technical Conference 2022: 37-52.) combines TEE technology with deduplication and lossless compression. It takes outsourced cloud storage scenarios into account, allowing third-party clouds to receive plaintext data from users, perform data deduplication and encryption within the TEE in the cloud, and further perform lossless compression on unique blocks to reduce storage overhead, thus realizing a secure and efficient file storage system.

[0006] Nevertheless, existing technologies do not consider combining TEE technology with incremental compression to further compress files while ensuring data security. Furthermore, simply implementing existing incremental compression technology in a trusted execution environment has the following drawbacks: (1) Limited shielded execution environment space. Incremental compression technology requires the system to read the base block into memory to incrementally encode the target block. Implementing incremental compression in a trusted execution environment will further require the system to load the base block into the trusted execution environment. However, the size of the shielded execution environment provided by the TEE is often limited (for example, the Intel GSX V1 only provides 256MiB of space). Using space exceeding the environment size will result in expensive paging overhead, making it impossible for the system to cache a large number of base blocks in the TEE; (2) Expensive context switching overhead. TEEs typically provide relevant interfaces (such as ECall and OCall provided by Intel SGX) to enable interaction between the shielded execution environment and external unprotected memory. The limited shielded execution environment space (i.e., Challenge 1) forces the system to continuously load base blocks into the TEE through the interface during incremental compression. However, the interface functions involve expensive hardware operations, resulting in significant context switching overhead and reduced system efficiency with each call.

[0007] Secondly, existing work (see Zhang Y, Xia W, Feng D, et al. Finesse: Fine-Grained Feature Locality based Fast Resemblance Detection for Post-Deduplication Delta Compression. 17th USENIX Conference on File and Storage Technologies. 2019) indicates that incremental compression technology can improve file compression efficiency by several times compared to deduplication and lossless compression, further saving the physical space required for data storage. Current file storage systems for trusted execution environments only implement deduplication and local compression technologies, but do not further consider incremental compression technology. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a bidirectional incremental compression method for trusted execution environments.

[0009] The objective of this invention is achieved through the following technical solution: a bidirectional incremental compression method for a trusted execution environment, used in a file storage system, wherein the file storage system includes a client and a cloud. The client provides the file to be stored, divides the file into blocks, and packages n plaintext data blocks M into a batch B and uploads it to the cloud. The cloud configures a threshold t, deploys a trusted execution environment, calculates the feature set F={f0,f1,f2} of the plaintext data block M, and performs deduplication, incremental compression, and lossless compression. The bidirectional incremental compression method includes the following steps: S1: Configure threshold t in the cloud, set the mapping between the unique block and the fingerprint of the feature index record, set the mapping between the data block to be subjected to backward incremental compression and the corresponding basic block of the backward index record, and set the mapping between the basic block and the set of incremental blocks based on the corresponding basic block of the incremental index record. S2: The cloud performs data deduplication for all data blocks in batch B, deletes the data blocks already stored in the system in batch B, and retains the remaining q unique data blocks M; S3: The feature set {F} of the remaining q unique data blocks M is calculated sequentially in the cloud; S4: Access the feature index in the cloud to find the base block of the remaining q unique data blocks M. The search method is as follows: If the feature set {F'} of the existing data block M' recorded in the feature index has the following relationship with {F}: If M' is considered to be a basic block of M; S5: The cloud determines that among the remaining q unique data blocks, p unique blocks exist in the corresponding basic block M'; S6: The cloud determines that the basic blocks M' corresponding to the p unique blocks are stored in r different containers respectively; S7: The cloud performs bidirectional incremental compression on p unique blocks that have a basic block M', namely forward incremental compression and backward incremental compression; S8: The cloud identifies the remaining qp unique blocks as base blocks, that is, the unique blocks that do not have base blocks as base blocks; the cloud performs lossless compression on these unique blocks and updates the feature information of the blocks to the feature index at the same time; S9: The cloud saves the basic blocks and incremental blocks into a container and performs persistent storage.

[0010] Preferably, step S7 includes the following sub-steps: S71: Calculate the intermediate parameter x. The formula for calculating the intermediate parameter x is: x = r ÷ n, where r is the number of containers distributed in the basic block M' corresponding to the p unique blocks, and n is the number of data blocks in a batch. S72: If x>t, where x is the calculated intermediate parameter and t is the cloud-preset threshold, then the cloud incrementally compresses p unique blocks based on the existing basic block M', i.e., forward incremental compression; and updates the incremental block and basic block information in the incremental index; otherwise, the cloud records the data block and the corresponding basic block in the backward index, preparing for subsequent execution of backward incremental compression. S73: The cloud reads the backward index. For the data block M and the corresponding base block M' recorded in the backward index, the cloud performs lossless decompression of M'. Then, the cloud queries the incremental index and loads the stored incremental block set {M''} which is incrementally compressed based on the base block M'. The incremental blocks in {M''} are incrementally decompressed based on M' to restore the original data. The cloud incrementally compresses the corresponding base blocks M' and {M''} based on these p unique blocks M, and uses M as the new base block, i.e., backward incremental compression. The feature index and incremental index are updated, and finally the backward index record is cleared. S74: Deleting the original incremental data {M''} in the incremental compression based on the old base block M' in the cloud; The threshold t is a settable value, which is read from the configuration file by the cloud during initialization. The size of t can be set according to the actual situation.

[0011] Preferably, in S1, the threshold t is 0.03 by default.

[0012] Preferably, the batch size B is 128 by default.

[0013] Preferably, all backward incremental compression operations are performed offline, i.e., on critical write paths that are not file storage.

[0014] Preferably, there are multiple clients.

[0015] The beneficial effects of this invention are: 1) Combining incremental compression with deduplication and local compression further reduces redundancy between similar data blocks and improves the storage efficiency of the file system.

[0016] 2) By using batch processing and bidirectional incremental compression methods, the problem of decreased incremental compression storage efficiency caused by the limitations of the shielded execution environment is mitigated, and file writing performance is improved.

[0017] 3) Performing backward incremental compression operations on non-critical write paths avoids the computational delay caused by backward incremental compression on critical write paths, thus improving file performance.

[0018] 4) Allows users to set the threshold t themselves to meet the needs of various workloads.

[0019] 5) By changing the base block of {M''} during backward incremental compression, the additional computational overhead incurred when restoring {M''} can be avoided. Attached Figure Description

[0020] Figure 1 The diagram illustrates the file writing speed of the method of this invention compared to the other three existing methods for four datasets. Figure 2 This diagram illustrates the data compression ratio of the method of this invention compared to three other existing methods for four datasets. Detailed Implementation

[0021] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] A bidirectional incremental compression method for Trusted Execution Environments (TEEs) is proposed. This method aims to address the issue of existing technologies failing to consider combining incremental compression with TEE technology and to mitigate the efficiency degradation caused by shielding the execution environment. Primarily applied to cloud storage, this method improves data storage efficiency and reduces data storage overhead while ensuring data security. Specifically, a client-side application supporting data read and write is deployed on the cloud storage client side; a Trusted Execution Environment (TEE) and persistent storage media are deployed in the cloud for secure data compression and persistent data storage.

[0023] See Figures 1-2This invention provides a technical solution: a bidirectional incremental compression method for a trusted execution environment, used in a file storage system. The file storage system includes a client and a cloud. The client provides the file to be stored, divides the file into blocks, and packages n plaintext data blocks M into a batch B, which is then uploaded to the cloud. The cloud configures a threshold t, deploys a trusted execution environment, calculates the feature set F={f0,f1,f2} of the plaintext data blocks M, and performs deduplication, incremental compression, and lossless compression. The bidirectional incremental compression method includes the following steps: S1: Configure threshold t in the cloud, set the mapping between the unique block and the fingerprint of the feature index record, set the mapping between the data block to be subjected to backward incremental compression and the corresponding basic block of the backward index record, and set the mapping between the basic block and the set of incremental blocks based on the corresponding basic block of the incremental index record. S2: The cloud performs data deduplication for all data blocks in batch B, deletes the data blocks already stored in the system in batch B, and retains the remaining q unique data blocks M; S3: The feature set {F} of the remaining q unique data blocks M is calculated sequentially in the cloud; S4: Access the feature index in the cloud to find the base block of the remaining q unique data blocks M. The search method is as follows: If the feature set {F'} of the existing data block M' recorded in the feature index has the following relationship with {F}: If M' is considered to be a basic block of M; S5: The cloud determines that among the remaining q unique data blocks, p unique blocks exist in the corresponding basic block M'; S6: The cloud determines that the basic blocks M' corresponding to the p unique blocks are stored in r different containers respectively; S7: The cloud performs bidirectional incremental compression on p unique blocks that have a basic block M', namely forward incremental compression and backward incremental compression; S8: The cloud identifies the remaining qp unique blocks as base blocks, that is, the unique blocks that do not have base blocks as base blocks; the cloud performs lossless compression on these unique blocks and updates the feature information of the blocks to the feature index at the same time; S9: The cloud saves the basic blocks and incremental blocks into a container and performs persistent storage.

[0024] In some embodiments, S7 further includes the following sub-steps: S71: Calculate the intermediate parameter x. The formula for calculating the intermediate parameter x is: x = r ÷ n, where r is the number of containers distributed in the basic block M' corresponding to the p unique blocks, and n is the number of data blocks in a batch. S72: If x>t, where x is the calculated intermediate parameter and t is the cloud-preset threshold, then the cloud incrementally compresses p unique blocks based on the existing basic block M', i.e., forward incremental compression; and updates the incremental block and basic block information in the incremental index; otherwise, the cloud records the data block and the corresponding basic block in the backward index, preparing for subsequent execution of backward incremental compression. S73: The cloud reads the backward index. For the data block M and the corresponding base block M' recorded in the backward index, the cloud performs lossless decompression of M'. Then, the cloud queries the incremental index and loads the stored incremental block set {M''} which is incrementally compressed based on the base block M'. The incremental blocks in {M''} are incrementally decompressed based on M' to restore the original data. The cloud incrementally compresses the corresponding base blocks M' and {M''} based on these p unique blocks M, and uses M as the new base block, i.e., backward incremental compression. The feature index and incremental index are updated, and finally the backward index record is cleared. S74: Deleting the original incremental data {M''} in the incremental compression based on the old base block M' in the cloud; The threshold t is a settable value, which is read from the configuration file by the cloud during initialization. The size of t can be set according to the actual situation.

[0025] In some embodiments, the threshold t in S1 is 0.03 by default.

[0026] In some embodiments, the batch size B is 128 by default.

[0027] In some embodiments, all backward incremental compression operations are performed offline, i.e., on critical write paths that are not file storage.

[0028] In some embodiments, there are multiple clients.

[0029] This embodiment will compare and analyze the bidirectional incremental compression method (hereinafter referred to as ShieldReduce) for trusted execution environments implemented based on the method of the present invention with existing redundancy reduction methods. Specifically, the following schemes are considered in the comparative analysis: (1) DEBE, a deduplication method for trusted execution environments, which further uses local compression technology to compress unique blocks; (2) ForwardDelta, a method that loads the container of the corresponding basic block into the shielded execution environment to perform forward incremental compression, without considering backward incremental compression; (3) SecureMeGA, a method that only loads containers containing a large number of basic blocks, and skips the incremental compression of data blocks whose basic blocks are not in these containers.

[0030] This embodiment will verify the effectiveness of the ShieldReduce method of the present invention based on four different types of datasets: (1) Linux dataset, including 210 version snapshots, containing a total of 187 GiB of logical data; (2) Web dataset, including 102 Sina News website backups, containing a total of 277.1 GiB of logical data; (3) Docker dataset, including 96 Cassandra image snapshots downloaded from Docker Hub, containing a total of 31.9 GiB of logical data; (4) SimOS dataset, including 30 snapshots generated from CentOS7 virtual disk images, each snapshot being modified from the previous snapshot, containing a total of 240 GiB of logical data. The four datasets used in this embodiment cover various types of workloads, including source code (Linux dataset), binary snapshots (Docker dataset), website backups (Web dataset), and operating system images (SimOS dataset).

[0031] The verification metrics in this embodiment include: (1) File write speed (i.e., Upload), which is the logical size of the file to be stored divided by the time required to store the file by each data reduction method. The faster the file write speed, the higher the storage performance of the method. This metric does not consider the impact of client hard disk I / O; (2) Data compression ratio (i.e., ReductionRatio), which is the ratio of the total logical size of the file to the physical space required to store the file by each data reduction method. The higher the data compression ratio, the higher the redundancy reduction efficiency of the method and the better the storage efficiency.

[0032] Figures 1-2 The analysis results for file write speed and data compression ratio of the embodiments are shown separately. Specifically, for file write speed, DEBE has the highest file write performance, achieving a speedup of 1.1-1.6 times compared to the method proposed in this invention (i.e., ShieldReduce). This is because DEBE does not require incremental compression. Meanwhile, ShieldReduce improves file write performance by 1.1-5.9 times compared to ForwardDelta, because this invention proposes a bidirectional incremental compression method. Finally, ShieldReduce and SecureMeGA have comparable file storage speeds because the latter trades data compression ratio for file write speed.

[0033] Regarding data compression ratio, ShieldReduce, ForwardDelta, and SecureMeGA achieve higher data compression ratios than DEBE because these methods eliminate redundant data between similar data blocks through incremental compression techniques. ShieldReduce achieves higher data compression ratios than SecureMeGA in all four datasets. Even in the Web dataset, the method proposed in this invention achieves a data compression ratio exceeding SecureMeGA by 6.2 times, because SecureMeGA skips some incremental compression operations. Finally, compared to ForwardDelta, ShieldReduce achieves similar data compression ratios, and even in Web and SimOS datasets, ShieldReduce improves upon ForwardDelta by 3.0% and 0.5%, respectively.

[0034] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A bidirectional incremental compression method for trusted execution environments, characterized in that: A bidirectional incremental compression method for a file storage system, the file storage system comprising: a client and a cloud, wherein the client provides the file to be stored, divides the file into blocks, and packages n plaintext data blocks M into a batch B and uploads it to the cloud; the cloud configures a threshold t, deploys a trusted execution environment, calculates the feature set F={f0,f1,f2} of the plaintext data block M, and performs deduplication, incremental compression, and lossless compression; the bidirectional incremental compression method includes the following steps: S1: Configure threshold t in the cloud, set the mapping between the unique block and the fingerprint of the feature index record, set the mapping between the data block to be subjected to backward incremental compression and the corresponding basic block of the backward index record, and set the mapping between the basic block and the set of incremental blocks based on the corresponding basic block of the incremental index record. S2: The cloud performs data deduplication for all data blocks in batch B, deletes the data blocks already stored in the system in batch B, and retains the remaining q unique data blocks M; S3: The feature set {F} of the remaining q unique data blocks M is calculated sequentially in the cloud; S4: Access the feature index in the cloud to find the base block of the remaining q unique data blocks M. The search method is as follows: If the feature set {F'} of the existing data block M' recorded in the feature index has the following relationship with {F}: If M' is considered to be a basic block of M; S5: The cloud determines that among the remaining q unique data blocks, p unique blocks exist in the corresponding basic block M'; S6: The cloud determines that the basic blocks M' corresponding to the p unique blocks are stored in r different containers respectively; S7: The cloud performs bidirectional incremental compression on p unique blocks that have a basic block M', namely forward incremental compression and backward incremental compression; S8: The cloud identifies the remaining qp unique blocks as base blocks, that is, the unique blocks that do not have base blocks as base blocks; the cloud performs lossless compression on these unique blocks and updates the feature information of the blocks to the feature index at the same time; S9: The cloud saves the basic blocks and incremental blocks into a container and performs persistent storage.

2. The bidirectional incremental compression method for trusted execution environments according to claim 1, characterized in that: S7 includes the following sub-steps: S71: Calculate the intermediate parameter x. The formula for calculating the intermediate parameter x is: x = r ÷ n, where r is the number of containers distributed in the basic block M' corresponding to the p unique blocks, and n is the number of data blocks in a batch. S72: If x>t, where x is the calculated intermediate parameter and t is the cloud-preset threshold, then the cloud incrementally compresses p unique blocks based on the existing basic block M', i.e., forward incremental compression; and updates the incremental block and basic block information in the incremental index; Otherwise, the cloud records the data block and the corresponding base block in the backward index, preparing for subsequent backward incremental compression; S73: The cloud reads the backward index. For the data block M and the corresponding base block M' recorded in the backward index, the cloud decompresses M' without loss. Then, the cloud queries the incremental index and loads the stored incremental block set {M''} which is incrementally compressed based on the base block M'. The incremental blocks in {M''} are incrementally decompressed based on M' to restore the original data. The cloud incrementally compresses the corresponding base blocks M' and {M''} based on these p unique blocks M, and uses M as the new base block, i.e. backward incremental compression; it also updates the feature index and incremental index, and finally clears the backward index record; S74: Deleting the original incremental data {M''} in the incremental compression based on the old base block M' in the cloud; The threshold t is a settable value, which is read from the configuration file by the cloud during initialization. The size of t can be set according to the actual situation.

3. The bidirectional incremental compression method for trusted execution environments according to claim 1 or 2, characterized in that: In S1, the threshold t is set to 0.03 by default.

4. The bidirectional incremental compression method for trusted execution environments according to claim 1 or 2, characterized in that: The default size of batch B is 128.

5. The bidirectional incremental compression method for trusted execution environments according to claim 1 or 2, characterized in that: All backward incremental compression operations are performed offline, i.e., on critical write paths that are not file storage.

6. The bidirectional incremental compression method for trusted execution environments according to claim 1 or 2, characterized in that: There are multiple clients.

Images