Secure Encryption Key-Value Data Storage Method and System
By redesigning the key-value storage system within the SGX security zone, storing keys and values separately and utilizing hash algorithms, the problem of high disk read/write overhead in existing technologies is solved, achieving efficient and secure key-value storage and improving performance and security.
Patent Information
- Application Number
- CN202311197959.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-18
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-09-18
AI Technical Summary
The existing SGX security zone implementation of key-value pair storage has excessive disk read/write overhead and the security attributes cannot be extended to the entire key-value storage data, resulting in reduced query performance.
By adopting the idea of trading space for time, the architecture of the secure encrypted key-value storage system is redesigned, with keys and values stored separately. External calls are used to reduce the scale of direct I/O operations within the Enclave, and symmetric encryption and hash algorithms are used to ensure data security and integrity.
It effectively reduces the scale of direct I/O operations within the Enclave, improves query performance by approximately 6 times, ensures data security, prevents adversaries from obtaining sensitive information, and improves write performance by 2.5-9.4 times and read performance by 2.6-6.8 times.
Smart Images

Figure CN117216814B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a technology in the field of information security, specifically a secure encrypted key-value pair data storage method and system. Background Technology
[0002] SGX (Software Guard Extension), as a typical Trusted Execution Environment (TEE), utilizes hardware-protected enclaves to safeguard data security. In cloud service systems, the Log-Structured Merge Tree (LSM-Tree) first stores data written to key-value stores sequentially in an in-memory table. When the size of the in-memory table reaches a certain threshold, a background thread is triggered to dump the data to disk, forming a Sorted String Table (SSTable). Existing solutions using SGX to implement secure encrypted key-value storage employ optimized direct I / O libraries for writing to and reading from disk. However, as the amount of stored data increases, I / O within the enclave becomes a bottleneck, introducing at least four times the additional latency and resulting in reduced query performance. Summary of the Invention
[0003] This invention addresses the problems of excessive disk read / write overhead and the inability to extend security attributes to the entire key-value storage data in existing SGX secure zones. It proposes a secure encrypted key-value storage method and system, which redesigns the structure of the secure encrypted key-value storage system based on the idea of trading space for time. It utilizes external calls to store keys and values separately, effectively reducing the scale of direct I / O operations within the Enclave.
[0004] This invention is achieved through the following technical solution:
[0005] This invention relates to a secure encrypted key-value pair data storage method, comprising: a secure write process and a secure read process.
[0006] The secure write process specifically includes: after the client establishes secure communication with the secure encrypted key-value data storage system, it sends the key-value data and the corresponding operation to the storage engine. The storage engine encrypts the key-value data using a symmetric encryption key and writes all insertion-related information into the write-ahead log and optimizes the structure of the log entries. When writing to the write-ahead log, the storage engine performs integrity verification on the structured log. Once it confirms that the log entry has been successfully written to the write-ahead log, it notifies the log counter to increment. Finally, the key-value data is officially written to the memory table.
[0007] All the insertion-related information mentioned refers to: encrypted key-value data and the current value of the log counter; data used to restore memory in the event of an unexpected system restart.
[0008] Preferably, when inserting into the second-level storage of the application, the storage engine additionally writes the value hash as part of the data into the write-ahead log.
[0009] The value hash mentioned refers to the result of calculating the value written during a secure write process using the SHA256 algorithm.
[0010] The aforementioned structural optimization refers to the following: each log entry includes a log header and a data portion, wherein the log header includes a message authentication code for identity authentication and ensuring data integrity, the data length, the value of the log counter, and the operation type, and the message authentication code is obtained through the CMAC algorithm.
[0011] The message authentication code is generated based on the message authentication key each time a write-ahead log is written. The message authentication key is generated during the initialization of the secure storage area.
[0012] The operation types mentioned refer to the three operations of adding, deleting, and modifying in the key-value storage system.
[0013] Preferably, the symmetric encryption key and message authentication code are both subjected to SGX sealing and persistence processing.
[0014] The log counter is globally unique and monotonically increasing throughout the system, while log entries can only be appended and not modified. Therefore, it is possible to determine whether the log file has been tampered with by checking the counter values of adjacent entries.
[0015] Preferably, before writing, the storage engine determines whether the length of the encrypted value itself is comparable to the hash result. If so, it does not use second-level storage and directly stores the encrypted value; otherwise, it writes it to memory through second-level storage.
[0016] The secure read process specifically includes:
[0017] ① The storage engine uses a symmetric key to encrypt the query key submitted by the user;
[0018] ② The storage engine searches in the memory enclave to see if the key is stored. Specifically, it first searches the second-level cache. If a match is found, no verification is needed, and the value is returned directly because the data written to the second-level cache satisfies the security properties. Otherwise, it queries the memory table. If a match is found, it executes step ③; otherwise, it executes step ④.
[0019] ③ When the data in the memory table is not using second-level storage, the data is encrypted. The storage engine only needs to decrypt the encrypted value and return it to the client, because an adversary cannot enter the Enclave to tamper with the data in the memory table. When the data in the memory table is using second-level storage, the storage engine reads the corresponding encrypted value from the second-level storage based on the value hash and attempts to decrypt the read data. When decryption is successful, it means that the value was written by the current client. The engine then compares whether the hash of the decrypted value is consistent with the hash of the value in the memory table. Only the value that passes the verification can be written to the second-level cache and returned to the client.
[0020] ④ If the target key is not found in memory, the storage engine counts all layers that might contain the target key, and then performs a binary search on each layer from top to bottom. Before further searching the contents of the sorted string table file found by the binary search, the storage engine checks whether the file's hash value matches the version information stored in the Enclave, thus ensuring the security of all data stored in the sorted string table. If the first matching target key is found during the search of the sorted string table, the search stops and the corresponding value is returned. Then, depending on whether the value enables second-level storage, it is decrypted according to step ③ and returned to the client; otherwise, it means that the key-value pair does not exist in the key-value storage system. Technical effect
[0021] This invention implements a key-value separation dual-layer storage model in the Enclave; it employs a second-layer caching mechanism to eliminate the redundant overhead of accessing hot data in the dual-layer storage model; and it provides a read / write algorithm that ensures the security of the entire key-value storage system. Compared to existing technologies, this invention's dual-layer storage model, which utilizes a direct I / O library, achieves approximately a 6x performance improvement; it reduces the number of external calls when querying hot data, while eliminating the overhead of multiple repeated decryptions; user data is stored in encrypted form in the cloud, and an integrity verification mechanism is added, preventing adversaries from obtaining any valid information about the stored data from external calls, including any valid information about the stateless data protected by this invention (i.e., key-value data) and any valid information about the stateful data protected by this invention (i.e., write-ahead logs and metadata manifests). Attached Figure Description
[0022] Figure 1 and Figure 2 This is a schematic diagram of the read / write method of the present invention;
[0023] Figure 3 This is a schematic diagram of the secure encrypted key-value pair data storage system of the present invention;
[0024] Figure 4 This is a schematic diagram of a two-tier storage system.
[0025] Figure 5 This is a schematic diagram of structural optimization. Detailed Implementation
[0026] like Figure 3 As shown in the figure, this embodiment relates to a secure encrypted key-value data storage system, including: a secure storage area (Enclave) and a second-level internal storage located in memory, and a sorted string table and a second-level external storage located externally. The secure storage area includes a validator, a second-level cache, a storage engine, and a memory table. The storage engine handles all interactions with the client and manages the entire log structure merging tree, storing data in the memory table or the second-level storage after encryption with a symmetric key. The validator verifies the log files on the disk according to the message authentication key, and after successful verification, hands the logs over to the storage engine for further recovery. The second-level cache caches recently accessed values from the second-level storage to the secure storage area, reducing the number of external calls during the read process when hot data exists. The second-level internal storage is the memory part of the second-level storage, while the second-level external storage is deployed on an untrusted disk as a persistent component to ensure that data is not lost. Together, they form a second-level storage that effectively improves the performance of key-value storage.
[0027] The symmetric encryption key is generated during the initialization of the secure storage area.
[0028] The data in the second-level cache can be stored in plaintext, thereby eliminating the overhead of repeated decryption when reading the same key each time.
[0029] The technical details / improvements (if any) of the verifier are as follows: The verifier first obtains the message authentication key, then performs integrity verification on each record in the log file, and at the same time verifies whether the value of the counter increases with each record.
[0030] The storage engine includes: a storage engine directly associated with a verifier, an encryption / decryption unit, and a counter; wherein: during the recovery of the key-value data storage system, the verifier transmits the successfully verified logs to the storage engine; the storage engine reads the logs one by one, restores the metadata list to the system metadata in the secure storage area, and restores the write-ahead logs to the memory table; the storage engine encrypts and decrypts the data through the encryption / decryption unit, and then stores it in the memory table or returns the result to the client; the storage engine operates and maintains the counter, incrementing the counter each time a log is successfully written.
[0031] like Figure 1 and Figure 2 The diagram illustrates the read / write method for the secure encrypted key-value pair data storage system described above in this embodiment, including a secure write process and a secure read process.
[0032] like Figure 4As shown, the two-tier storage refers to using value hashes as the keys and encrypted values as the values of its basic storage units. Its advantage is that even if the two-tier storage is completely exposed to an untrusted space, the stored value hashes and encrypted values will not reveal any information about the real values. In the first-tier storage within the trusted space, encrypted keys are used as the keys and value hashes as the values of their basic storage units. The association between the first-tier and second-tier storage is achieved through external calls; that is, the first-tier storage within the Enclave performs secondary addressing via value hashes, while disk I / O is performed in the untrusted space to search for the corresponding encrypted value. Although the two-tier storage design introduces additional queries and storage space, it effectively reduces the scale of I / O within the Enclave. The two-tier storage mechanism transforms I / O within the Enclave into a combination of external calls and untrusted space I / O. Assuming a sorted string table file stores 16 bytes of keys and 512 bytes of values, and the maximum file size is 2MB, then approximately 4000 key-value pairs can be stored in a single file. However, if a two-level storage is used, storing keys and value hashes in the sorted string table only requires 32 bytes. In this case, approximately 43,000 entries can be stored in a single file, improving space utilization by about 10 times. The I / O scale within the Enclave is inversely proportional to the storage capacity of the sorted string table for key-value pairs. Therefore, in scenarios with long values, the two-level storage model can optimize a significant portion of the Enclave's I / O.
[0033] Through specific practical experiments, an experimental environment was set up on a machine equipped with an Intel i5-8250U processor (8 cores, 1.6GHz) and 8GB of memory. The server operating system was 64-bit Ubuntu 20.04.4 LTS, Linux kernel version 5.15.0-48, and SGX OOT driver version 2.11 and SDK and platform software suite version 2.16 were deployed. In the experiment, it was assumed that in the key-value pairs stored in the cloud, the key length was much shorter than the value length. Furthermore, this scheme considers a powerful adversary with privileged access to the operating system's software stack and hardware infrastructure, enabling them to monitor and tamper with the server's memory data, disk read / write operations, and system software. However, the adversary cannot access the processor hardware-protected components, i.e., the content protected by SGX. To test the performance of this embodiment, two workloads with different distributions were set up in the experiment: uniform distribution and skewed distribution, with each workload containing 1 million data entries.
[0034] The uniformly distributed workload refers to the fact that when inserting key-value pairs, the values of the keys are uniformly distributed, and the probability of requesting each key is the same during a query.
[0035] The skewed workload refers to a situation where, during key-value pair insertion, the key values exhibit a skewed distribution with a skewness of 0.99, and the requested keys also exhibit a skewed distribution with a skewness of 0.99 during queries. Furthermore, based on the proportion of read operations, the workload is subdivided into three types: RD_50 has a 50 / 50 ratio of read and write operations; RD_95 has 95% read operations and 5% write operations; and RD_100 involves only read operations.
[0036] The results of the experiment are shown in Table 1 and Table 2.
[0037] Table 1. Performance comparison of various technical solutions under uniformly distributed workload;
[0038] Table 2 shows the performance comparison results of each technical solution under skewed distributed workload.
[0039] The technical solutions compared in the table refer to the following: Speicher is an existing secure encrypted key-value data storage solution; this solution is based on LevelDB, adds SGX protection, and is further divided into three specific implementations: a basic encryption solution without secondary storage, a solution with secondary storage, and a solution with secondary storage and secondary caching. Compared with the existing Speicher technology, this invention reduces the size of the SSTable to 1 / 8 of that in Speicher by storing keys and values in two layers; under uniformly distributed workloads, the read performance of this solution is 2.6-2.8 times better than Speicher, and the write performance is 2.5-9.4 times better; under skewed RD_50, RD_95, and RD_100 workloads, its performance is 3.2-6.8 times better than Speicher.
[0040] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.
Claims
1. A secure encrypted key-value pair data storage method, characterized in that, include: Secure write and secure read processes; The secure write process includes: after the client establishes secure communication with the secure encrypted key-value data storage system, it sends the key-value data and corresponding operations to the storage engine. The storage engine encrypts the key-value data using a symmetric encryption key and writes all insertion-related information into the write-ahead log and optimizes the log entries. When writing to the write-ahead log, the storage engine performs integrity verification on the optimized log. Once it confirms that the log entries have been successfully written to the write-ahead log, it notifies the log counter to increment. Finally, the key-value data is officially written to the memory table. The secure read process includes: 1) The storage engine uses a symmetric key to encrypt the query key submitted by the user; 2) The storage engine searches in the memory enclave to see if a matching key is stored. Specifically, it first searches the second-level cache. If a match is found, no verification is needed, and the value is returned directly because the data written to the second-level cache satisfies the security properties. Otherwise, it queries the memory table. If a match is found, it executes step 3); otherwise, it executes step 4). 3) When the data in the memory table is not using Layer 2 storage, the data is encrypted. The storage engine only needs to decrypt the encrypted value and return it to the client, because an adversary cannot access the Enclave to tamper with the data in the memory table. When the data in the memory table is using Layer 2 storage, the storage engine reads the corresponding encrypted value from the Layer 2 storage based on the value hash (the result of the SHA256 algorithm calculated from the value written during the secure write process). It then attempts to decrypt the read data. If decryption is successful, it indicates that the value was written by the current client. The engine then compares the hash of the decrypted value with the hash of the value in the memory table. Only values that pass the full verification are written to the Layer 2 cache and returned to the client. 4) If the target key is not found in memory, the storage engine counts all layers that may contain the target key, and then performs a binary search on each layer from top to bottom. Before further searching the contents of the sorted string table file found by the binary search, the storage engine checks whether the hash value of the file is consistent with the version information stored in the Enclave. Thus, all data stored in the sorted string table is secure. If the first matching target key is found during the search of the sorted string table, the search stops and the corresponding value is returned. Then, depending on whether the second-level storage is enabled, the value is decrypted according to step 3) and returned to the client; otherwise, it means that the key-value pair does not exist in the key-value storage system. The aforementioned structural optimization refers to the following: each log entry includes a log header and a data portion, wherein the log header includes a message authentication code for identity authentication and ensuring data integrity, the data length, the value of the log counter, and the operation type, and the message authentication code is obtained through the CMAC algorithm; The message authentication code is generated based on the message authentication key each time a write-ahead log is written. The message authentication key is generated when the secure storage area is initialized. The operation types mentioned refer to the three operations of adding, deleting, and modifying in the key-value storage system; The aforementioned two-layer storage refers to a storage system that uses value hashes as the keys and encrypted values as the values. The second-level cache caches recently accessed values from the second-level storage to the secure storage area, reducing the number of external calls during the read process when hot data exists.
2. The secure encrypted key-value pair data storage method according to claim 1, characterized in that, All the insertion-related information mentioned refers to: encrypted key-value data and the current value of the log counter; data used to restore memory in the event of an unexpected system restart.
3. The secure encrypted key-value pair data storage method according to claim 2, characterized in that, When inserting into the application's Tier 2 storage, the storage engine additionally writes the value hash as part of the data into the write-ahead log.
4. The secure encrypted key-value pair data storage method according to claim 1, characterized in that, The log counter is globally unique and monotonically increasing throughout the system, while log entries can only be appended and not modified. The log file is determined to have been tampered with by checking the counter values of adjacent entries.
5. The secure encrypted key-value pair data storage method according to claim 1, characterized in that, Before writing, the storage engine determines whether the length of the encrypted value is equal to the hash result. If so, it will not use second-level storage and will directly store the encrypted value; otherwise, it will write the encrypted value into memory through second-level storage.
6. A secure encrypted key-value pair data storage system implementing the method of any one of claims 1-5, characterized in that, include: The system consists of a secure storage area and a second-level internal storage in memory, as well as a sorted string table and a second-level external storage located on the outside. The secure storage area contains a validator, a second-level cache, a storage engine, and a memory table. The storage engine handles all interactions with the client and manages the entire log structure merging tree, storing data in the memory table or the second-level storage after encrypting it with a symmetric key. The verifier verifies the log file on the disk based on the message authentication key. After verification, the log is handed over to the storage engine for further recovery. The second-level cache caches recently accessed values from the second-level storage to the secure storage area, reducing the number of external calls during the read process when hot data exists. The second-level internal storage is the memory part of the second-level storage, while the second-level external storage is deployed on an untrusted disk as a persistent component to ensure that data is not lost. Together, they form a second-level storage that effectively improves the performance of key-value storage. The symmetric encryption key is generated during the initialization of the secure storage area.
7. The secure encrypted key-value pair data storage system according to claim 6, characterized in that, The data in the second-level cache is stored in plaintext, eliminating the overhead of repeated decryption when reading the same key each time.
8. The secure encrypted key-value pair data storage system according to claim 6, characterized in that, The verification process refers to the following: the verifier first obtains the message authentication key, then performs integrity verification on each record in the log file, and at the same time verifies whether the value of the counter increases with each record.
9. The secure encrypted key-value pair data storage system according to claim 6, characterized in that, The storage engine includes: a storage engine directly associated with a verifier, an encryption / decryption unit, and a counter; wherein: during the recovery of the key-value data storage system, the verifier transmits the successfully verified logs to the storage engine; the storage engine reads the logs one by one, restores the metadata list to the system metadata in the secure storage area, and restores the write-ahead logs to the memory table; the storage engine encrypts and decrypts the data through the encryption / decryption unit, and then stores it in the memory table or returns the result to the client; the storage engine operates and maintains the counter, incrementing the counter each time a log is successfully written.
Citation Information
Patent Citations
SDN controller interface control method and system
CN116112195A
SGX-based secure high-availability key value storage method and system
CN116467733A
