Variable self-adaptive casual storage method for dynamic workload

Abstract

The embodiment of the invention relates to a variable self-adaptive casual storage method for a dynamic workload, and the method comprises the steps: determining a service sequence of each ORAM based on a planner, responding to a service conversion request message of a user, taking a RingORAM as an intermediary, carrying out the conversion between ConcuurORAM services and / or PathORAMs, and carrying out the conversion between the ConcuurORAM services and / or PathORAMs. A data bucket in a currently accessed ORAM client and metadata of the data bucket are refreshed and stored by using a two-time refresh algorithm, accessed data stored in an ORAM server are obtained, three representative ORAMs (PathORAM, RingORAM and ConcuurORAM) are taken as examples, a service conversion process is given, a variable ORAM in the conversion process can ensure original data confidentiality and carelessness of the ORAM, and the service conversion efficiency is improved. And no data leakage is generated.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of storage technology, and in particular to a variable adaptive oblivious storage method for dynamic workloads. Background Art

[0002] Encrypted databases can perform operations such as query and update on encrypted data without leaking data content information. They have been widely studied and concerned by academia and industry for a long time, and have been applied to fields such as medical care (encrypted electronic health records) and finance (anonymous blockchain systems such as Zero Cash). Existing solutions mainly focus on improving the query performance of encrypted databases or expanding their query functions. Among them, oblivious random access memory (ORAM) has been widely studied and applied in the field of encrypted databases due to its strong security protection. ORAM is a technology that can hide user access patterns, that is, hide the frequency and operation type of user access to data, and provide stronger security protection than data encryption.

[0003] Although ORAM has such strong security, its network communication overhead is still high for storage devices. For access to N data blocks, the most classic PathORAM will cause O(logN) communication overhead for each operation. In addition, ORAM inevitably generates nearly twice the redundant storage, which also poses a challenge to the server memory size. Such prohibitive access overhead limits the application of ORAM to a wider range of scenarios. In response to this, existing research is committed to improving the ORAM system structure or using new storage devices to improve its performance. For example, RingORAM uses the computing power of the server to achieve O(1) client online overhead; PS-ORAM applies ORAM to non-volatile memory (NVM) to obtain crash consistency and better system scalability. In addition, many studies also focus on expanding its application scenarios. For example, ConcurORAM maintains metadata on the server side for parallel access under asynchronous networks; Oblix uses ORAM and trusted execution environment (TEE) to build an oblivious ciphertext search system that supports dynamic updates. These ORAMs are designed for specific workloads and can serve the workloads efficiently.

[0004] However, in real life, workloads are often dynamically changing. Take a medical database as an example. When doctors directly access the database during daily consultations, they usually have no special latency or throughput requirements. However, in telemedicine or disaster backup, the database needs to process a large amount of data in real time, requiring lower latency and higher throughput. In addition, in daily consultations, doctors usually only search for items by keywords, but when performing medical analysis, more complex query operations such as sorting, judgment, and aggregation may be required. These workloads can be briefly summarized as: general storage to minimize maintenance costs, real-time updates to provide low operation latency, high throughput, and other specific functional requirements. These dynamic workloads impose diverse performance and functional requirements on ORAM, which cannot be effectively met by a single ORAM.

[0005] A direct way to deal with this dynamic nature is to maintain multiple ORAM instances and update all ORAMs simultaneously on each access, but this results in greater access overhead and increases data redundancy and maintenance costs. Another approach is to download and rebuild ORAM during different workloads, which, although able to adapt to dynamic workloads, also incurs large communication overhead and causes long service interruptions. Summary of the invention

[0006] In view of this, an embodiment of the present application proposes a variable adaptive oblivious storage method for dynamic workloads, which aims to ensure the original data confidentiality and obliviousness of ORAM without causing any data leakage.

[0007] To achieve the above-mentioned purpose, an embodiment of the present application provides a variable adaptive oblivious storage method for dynamic workloads, which is applied to a client of a variable ORAM, and the client has a planner. The variable adaptive oblivious storage method for dynamic workloads includes: in response to first configuration information input by a user, determining the service order of each ORAM based on the planner, wherein the ORAM represents oblivious memory, each of the ORAMs includes RingORAM, PathORAM and ConcurORAM, and each of the ORAMs has a data bucket and metadata of the data bucket, wherein the metadata of the data bucket is based on the maximum number of accesses to the data bucket. and the current access times; in response to the service conversion request message of the user, using the RingORAM as an intermediary, performing conversion between ConcurORAM services and / or PathORAM; in response to the pre-access message of the user, refreshing the first data bucket and the metadata of the first data bucket in the currently accessed ORAM client based on the refresh algorithm, obtaining the metadata of the second data bucket and the second data bucket, and in response to the access request message of the user, obtaining the accessed data stored in the ORAM server, refreshing the second data bucket and the metadata of the second data bucket based on the refresh algorithm, and obtaining the third data bucket and the metadata of the third data bucket.

[0008] Optionally, before performing conversion between ConcurORAM services and / or PathORAM using the RingORAM as an intermediary in response to the service conversion request message of the user, the variable adaptive oblivious storage method for dynamic workloads further includes: generating an encrypted database in response to content information input by the user, and sending the encrypted database to a server of the variable ORAM.

[0009] Optionally, each of the ORAMs has a client, a server, a server API and a client API, and in response to the user's service conversion request message, the RingORAM is used as an intermediary to complete the conversion between the ConcurORAM service and / or the PathORAM service, including: encrypting and processing the local cache data, and using virtual blocks to fill the local cache data to obtain the first synchronization metadata, uploading the first synchronization metadata to the ConcurORAM server, and using the ConcurORAM client API to download the first synchronization metadata and the second synchronization metadata, and restoring the local cache data based on the first synchronization metadata and the second synchronization metadata, respectively switching the ConcurORAM server API, the client API and the data structure to the RingORAM server API, the client API and the metadata of the data bucket to complete the conversion of the ConcurORAM service to the RingORAM service; respectively switching the RingORAM server API, the client API and the data structure to the PathORAM server API, the client API and the metadata of the data bucket to complete the conversion of the RingORAM service to the PathORAM service.

[0010] Optionally, in response to the user's pre-access message, refreshing the first data bucket and the metadata of the first data bucket in the currently accessed ORAM client based on the refresh algorithm to obtain the metadata of the second data bucket and the second data bucket includes: based on the user's pre-access message, obtaining the refresh mode of the currently accessed ORAM, wherein the refresh mode includes a periodic refresh mode and an immediate refresh mode; if it is a periodic refresh mode, traversing the data buckets in the ORAM, determining the data buckets that meet the refresh conditions, and setting the metadata of the data buckets that meet the conditions as initial values, and the expression of the initial value of the metadata of the data bucket is;

[0011] RecordMap[bid]=(0,lim b )

[0012] Among them, lim bIndicates the maximum number of accesses to a data bucket in a base ORAM client, where the base ORAM is RingORAM, bid indicates a data bucket in an ORAM client, and RecordMap indicates metadata of a data bucket in an ORAM client; if the previously accessed ORAM is in instant refresh mode, no operation is performed; the metadata values ​​of all the data buckets that meet the refresh conditions are updated from (cnt, lim) to (cnt+1, min(lim, lim′)) to obtain metadata of the second data bucket and the second data bucket, where lim′ indicates the maximum number of accesses to the data bucket before the refresh.

[0013] Optionally, in response to the user's access request message, the accessed data stored in the ORAM server is obtained, the second data bucket and the metadata of the second data bucket are refreshed based on the refresh algorithm, and the third data bucket and the metadata of the third data bucket are obtained, including: traversing all the second data buckets, if the ORAM currently being accessed is in periodic refresh mode, the metadata of all the second data buckets are reset to an initial value, if the ORAM currently being accessed is in instant refresh mode, the metadata of the second data bucket is reset to (1, 1), and the second data bucket after the metadata is reset is obtained; traversing the second data bucket after the metadata is reset again, if the ORAM currently being accessed is in periodic refresh mode, the second data buckets after the metadata is reset are reset to an initial value again, and if the ORAM currently being accessed is in instant access mode, no operation is performed.

[0014] Optionally, in response to the first configuration information input by the user, determining the service order of each ORAM based on the planner includes: in response to the first configuration information input by the user, determining the server-side monetary cost based on the planner, wherein the server-side monetary cost includes communication cost, server-side computing cost and server-side storage cost, and the first configuration information is the client workload order; determining the server-side parameters of the variable ORAM based on the server-side monetary cost; determining the first overhead model and the second overhead model of the client based on the server-side parameters of the variable ORAM; determining the minimum monetary cost of the client of each ORAM based on the first overhead model and the second overhead model of the client; and determining the service order of each ORAM based on the minimum monetary cost.

[0015] Optionally, determining the first overhead model and the second overhead model of the client based on the server parameters of the variable ORAM includes: determining the first overhead model based on the server parameters and data capacity parameters of the variable ORAM, wherein the expression of the first overhead model is:

[0016]

[0017] Among them, op represents the client operation, which can represent the query operation of a certain ORAM or the conversion operation between any two ORAMs. Lop represents the client operation delay, C cpu represents the computing capacity of the client, C down Indicates the download capacity of the client, C up Indicates the upload capacity of the client, n cpu Indicates the number of CPU cores on the server, P cpu Indicates the processing speed of a single CPU core on the server, P down Indicates the download speed of the server, P up Indicates the upload speed of the server, L net Indicates the network delay of the server, L IO represents the hardware delay of the server; based on the preset batch queue size and the delay of the client, the expression of the second overhead model is determined as:

[0018]

[0019] Among them, X op Indicates the throughput of the client, and BatchSize indicates the batch queue size of the client.

[0020] Optionally, after determining the service order of each ORAM based on the server currency and the client overhead, the variable adaptive oblivious storage method for dynamic workloads further includes: determining parameters of each ORAM based on the planner in response to the second configuration information input by the user.

[0021] Optionally, in response to the second configuration information input by the user, determining the parameters of each ORAM based on the planner includes: obtaining the size of the data bucket of each ORAM, the size of the data block in the data bucket, and the size of the data set to be stored; and determining the first bandwidth expansion parameter and the second bandwidth expansion parameter of the parameters of each ORAM based on the size of the data bucket, the size of the data block, and the size of the data set to be stored.

[0022] Optionally, determining the first bandwidth expansion parameter and the second bandwidth expansion parameter of the parameters of each ORAM based on the size of the data bucket, the size of the data block and the size of the data set to be stored includes: judging the size relationship between the data volume of the data set to be stored and the data volume of the data block; if the data volume of the data set to be stored is less than the data volume of the data block, calculating the first bandwidth expansion parameter of each ORAM by using a first formula, the first formula being:

[0023]

[0024] Wherein, Z represents the size of the data bucket, B represents the size of the data block, F represents the size of the data set to be stored in each ORAM, and M represents the total size of the data set to be stored; if the data volume of the data set to be stored is greater than the data volume of the data block, the second bandwidth expansion parameter of each ORAM is calculated using the second formula, and the second formula is:

[0025]

[0026] Among them, Z represents the size of the data bucket, B represents the size of the data block, and M represents the total size of the data set to be stored.

[0027] To achieve the above objectives, an embodiment of the present application also provides a variable adaptive oblivious storage method for dynamic workloads.

[0028] The embodiment of the present application proposes a variable adaptive oblivious storage method for dynamic workloads, which determines the service order of each ORAM based on the planner in response to the first configuration information input by the user, wherein ORAM represents oblivious memory, each ORAM includes RingORAM, PathORAM and ConcurORAM, and each ORAM has a data bucket and metadata of the data bucket, wherein the metadata of the data bucket is determined based on the maximum access number and the current access number of the data bucket; in response to the user's service conversion request message, the RingORAM is used as an intermediary to perform conversion between ConcurORAM services and / or PathORAM; in response to the user's pre-access message, the first data bucket and the metadata of the first data bucket in the currently accessed ORAM client are refreshed based on the refresh algorithm to obtain the second data bucket and the metadata of the second data bucket, in response to the user's access request message, the accessed data stored in the ORAM server is obtained, and the metadata of the second data bucket and the second data bucket in the ORAM client are refreshed based on the refresh algorithm to obtain the third data bucket and the metadata of the third data bucket. The present invention first designs a variable ORAM. Variable ORAM can be efficiently converted between different ORAM services. The present invention finds two potential leakage risks during the conversion process. The present invention takes three representative ORAMs (PathORAM, RingORAM and ConcurORAM) as examples to provide a service conversion process. Finally, the present invention provides a planner that is suitable for a variety of application scenarios. Variable ORAM has been strictly proven to be safe and can guarantee the original data confidentiality and inadvertence of ORAM, and no data leakage will occur during the conversion process. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1is a flow chart of a variable adaptive oblivious storage method for dynamic workloads provided in one embodiment of the present application;

[0030] Figure 2 It is a message flow diagram of a variable adaptive oblivious storage method for dynamic workloads provided in one embodiment of the present application;

[0031] Figure 3 It is an OST principle diagram of a variable adaptive oblivious storage method for dynamic workloads provided in one embodiment of the present application. DETAILED DESCRIPTION

[0032] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0033] An embodiment of the present application proposes a variable adaptive oblivious storage method for dynamic workloads, which is applied to an electronic device, wherein the electronic device can be a terminal or a server. This embodiment and the following embodiments are all described by taking the server as an example. The implementation details of the variable adaptive oblivious storage method for dynamic workloads proposed in this embodiment are described in detail below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0034] This method is a variable adaptive oblivious storage method for dynamic workloads, involving research fields such as memory (usually random access memory, RAM), oblivious RAM (Oblivious RAM, ORAM) and encrypted database (Encrypted Database), especially applying ORAM to dynamic workloads (Dynamic Workload).

[0035] Encrypted databases can perform operations such as query and update on encrypted data without leaking data content information. They have been widely studied and concerned by academia and industry for a long time, and have been applied to fields such as medical care (encrypted electronic health records) and finance (anonymous blockchain systems such as Zero Cash). Existing solutions mainly focus on improving the query performance of encrypted databases or expanding their query functions. Among them, oblivious random access memory (ORAM) has been widely studied and applied in the field of encrypted databases due to its strong security protection. ORAM is a technology that can hide user access patterns, that is, hide the frequency and operation type of user access to data, and provide stronger security protection than data encryption.

[0036] In real life, workloads are usually dynamically changing. Take medical databases as an example. When doctors directly access the database during daily consultations, there are usually no special latency or throughput requirements. However, in telemedicine or disaster backup, the database needs to process large amounts of data in real time, requiring lower latency and higher throughput. In addition, in daily consultations, doctors usually only search for items by keywords, but when performing medical analysis, more complex query operations such as sorting, judgment, and aggregation may be required. These workloads can be briefly summarized as: general storage to minimize maintenance costs, real-time updates to provide low operation latency, high throughput, and other specific functional requirements. These dynamic workloads impose diverse performance and functional requirements on ORAM, which cannot be effectively met by a single ORAM.

[0037] A direct way to deal with this dynamic nature is to maintain multiple ORAM instances and update all ORAMs simultaneously on each access, but this results in greater access overhead and increases data redundancy and maintenance costs. Another approach is to download and rebuild ORAM during different workloads, which, although able to adapt to dynamic workloads, also incurs large communication overhead and causes long service interruptions.

[0038] In order to solve the above problems, the present invention first designs a variable and adaptive ORAM framework (Versatile ORAM Framework, V-ORAM). V-ORAM can efficiently convert between different ORAM services. Secondly, the present invention finds two potential leakage risks in the conversion process and proposes an ORAM service transformation (ORAM Service Transformation, OST) protocol. The OST protocol can complete the conversion between ORAMs with a constant overhead that is independent of the total amount of ORAM data while ensuring security. The present invention takes three representative ORAMs (PathORAM, RingORAM and ConcurORAM) as examples to give a specific conversion process. Finally, the present invention provides a planner that can model the user's monetary cost and ORAM system performance. The planner is suitable for a variety of application scenarios, for example, selecting a suitable and most cost-effective ORAM according to the user's system performance requirements, or adjusting the server configuration according to performance requirements, and selecting the best ORAM parameters according to the user's data set.

[0039] Some important terms in this invention are explained as follows:

[0040] Oblivious random access memory: As the core content of the present invention, the present invention first briefly introduces the operation process of ORAM by taking PathORAM as an example. In PathORAM, the client maintains the encrypted data block (size is) in a data bucket, each data bucket contains Z data blocks, and then maintains the data bucket as a full binary tree and stores it in the server. Each data block is mapped to the path of the binary tree (the path refers to all data buckets from a leaf node to the root node). When accessing a data block, the client first reads the entire path where the data block is located, and saves all data blocks in the local cache (Stash). The data blocks in the Stash are then re-encrypted, shuffled and written back to the binary tree on the server side. This operation is called refresh (Eviction). PathORAM will randomly map the accessed data block to a new path after each access. In this way, the server will not be able to infer the accessed block based on the client's access operation.

[0041] RingORAM basic process: RingORAM is an ORAM that uses server-side computing. It maintains user-renewable virtual blocks inside the data bucket. Each time it is accessed, the virtual block is XORed with the target block, so only a single data block needs to be transmitted, achieving O(1) online communication overhead. In addition, in order to maintain a constant overhead, RingORAM changes its refresh operation to a periodic one, performing a refresh every A operations.

[0042] ConcurORAM basic process: ConcurORAM is an asynchronous ORAM structure with batch processing. It can support up to c concurrent accesses. It maintains metadata for synchronization on the server side, mainly including StashSet and DRLogSet. StashSet contains c entries, each of which is an encrypted Stash generated by an access, called TempStash. TempStash contains at most MasStashSize encrypted data blocks and at least c virtual blocks. DRLogSet is used to record the results of each access, including c data entries and c virtual blocks.

[0043] Virtual blocks and metadata: Some ORAMs maintain virtual blocks in data buckets, which are used for ORAM access operations. Each virtual block is initially marked as valid and is marked as invalid after being accessed once. Each virtual block can only be accessed once, and repeated access will cause leakage. In addition, ORAM needs to maintain metadata to track the validity and location of virtual blocks. RingORAM and ConcurORAM both maintain virtual blocks and metadata.

[0044] Instant refresh and periodic refresh: The refresh operations of different ORAMs are different. In the present invention, the type of refresh performed on each access is called instant refresh (InstantEviction), such as PathORAM. The type of refresh performed after a specific number of accesses is called periodic refresh (PeriodicEviction), such as RingORAM and ConcurORAM. Among them, periodic refresh usually maintains the above virtual blocks for periodic refresh, while instant refresh only accesses and refreshes real blocks.

[0045] The specific process of the variable adaptive oblivious storage method for dynamic workloads proposed in this embodiment can be as follows: Figure 1 As shown, reference Figure 1 , the method is applied to a client of a variable ORAM, the client has a planner, and the storage method may include:

[0046] S10, in response to the first configuration information input by the user, determining the service order of each ORAM based on the planner, wherein ORAM stands for Oblivious Memory, each ORAM includes RingORAM, PathORAM and ConcurORAM, each ORAM has a data bucket and metadata of the data bucket, wherein the metadata of the data bucket is determined based on the maximum access count and the current access count of the data bucket;

[0047] S20, in response to the user's service conversion request message, using RingORAM as an intermediary to convert between ConcurORAM services and / or PathORAM;

[0048] In an embodiment of the present invention, before step S20, the variable adaptive oblivious storage method for dynamic workloads may further include the following specific execution process:

[0049] S01. Generate an encrypted database in response to content information input by a user, and use a planner to send the encrypted database to a server of a variable ORAM.

[0050] Specifically, this step is the process of the client of the variable ORAM sending data to the server of the variable ORAM. This step is performed by the planner in the client of the variable ORAM. The planner generates an ORAM-based encrypted database based on the client input and uploads the database to the server of the variable ORAM. At the same time, the client generates the client and API of each ORAM locally, and the server of the variable ORAM generates the server and API of each ORAM accordingly. Each ORAM maintains the necessary state (State) on both the client and the server.

[0051] When the user wants to switch the service type, a conversion request is sent to the OST module, and the OST converts the API and status of each ORAM.

[0052] The present invention designs an efficient ORAM service conversion protocol. First of all, it is unrealistic to design a conversion protocol between any two ORAMs. Therefore, in order to reduce the design difficulty of the OST protocol, the present invention selects a special ORAM as the conversion intermediary, called base ORAM. Any ORAM is first converted to base ORAM during the conversion, and then converted from base ORAM to target ORAM. Therefore, OST only needs to design a set of conversion protocols for each ORAM. The conversion protocol is divided into two stages: metadata preparation and API switching. The present invention uses RingORAM as the conversion intermediary and provides a conversion algorithm for ConcurORAM and PathORAM.

[0053] As before, each ORAM has a client, a server, a server API, and a client API.

[0054] In an embodiment of the present invention, step S20 may include the following specific execution process:

[0055] S21, encrypt and process the local cache data, and use the virtual block to fill the local cache data to obtain the first synchronization metadata, upload the first synchronization metadata to the ConcurORAM server, and use the ConcurORAM client API to download the first synchronization metadata and the second synchronization metadata, and restore the local cache data based on the first synchronization metadata and the second synchronization metadata, and switch the ConcurORAM server API, client API and data structure to RingORAM server API, client API and data bucket metadata, respectively, to complete the conversion of ConcurORAM service to RingORAM service;

[0056] The client of Variable ORAM first encrypts the Stash (local cache) locally, and then fills the local cache with virtual blocks to obtain TempStash, that is, obtains the first synchronization metadata. Then upload TempStash to the server of ConcurORAM to obtain the second synchronization metadata, where the second synchronization metadata can include the latest Temp Stash and DRLogSet. The client of ConcurORAM can restore Stash according to the second synchronization metadata. Here the client only needs the latest TempStash because if the entire StashSet is restored, the refreshed blocks will be added to Stash again, causing the size of Stash to increase uncontrollably. DRLogSet requires an entire traversal to avoid missing previous data blocks. Finally, the client and server of ConcurORAM change the API and start the RingORAM service.

[0057] S22: Switch the server API, client API and data structure of RingORAM to the server API, client API and metadata of data bucket of PathORAM respectively, and complete the conversion of RingORAM service to PathORAM service.

[0058] The service conversion between RingORAM and PathORAM does not involve metadata on the server side, so there is no need to process metadata like ConcurORAM. Just switch the API directly. However, since RingORAM maintains virtual blocks in the bucket, PathORAM does not need virtual blocks. Therefore, the ORAM service conversion protocol OST also needs to deal with the bucket size of PathORAM during the access process. A direct solution is to retrieve the entire bucket together with the virtual blocks, and then write the same number of new virtual blocks back to the bucket. Although this method is safe and feasible, it will double the access overhead of PathORAM because the number of virtual blocks is usually close to the real blocks. OST hopes to maintain the original bucket size of PathORAM, that is, only access real data blocks. However, this access method will cause the server to detect the virtual blocks in the bucket, and then leak the client's access mode in subsequent accesses. Therefore, OST also needs to additionally deal with the leakage problem that exists during the conversion process.

[0059] The present invention points out two leakages in the conversion process and provides an efficient solution, namely the OST protocol. The OST protocol solves the leakage problem in the service conversion process through a data structure RecordMap and a refresh algorithm EvictRecord.

[0060] Specifically, the leakage problems existing in the conversion process include leakage 1 and leakage 2. Among them, leakage 1 (repeated access to invalid virtual blocks): Figure 2 As shown in the figure, when two different ORAMs with periodic refresh are switched, for example, RingORAM and ConcurORAM. If the metadata corresponding to each ORAM is roughly deleted and the metadata is regenerated for the subsequent ORAM, the subsequent ORAM will not be able to detect the previously accessed virtual blocks, which will cause the invalid virtual blocks to be accessed again and cause information leakage. Therefore, the metadata of the virtual blocks must be inherited.

[0061] Leak 2 (virtual block position leak): Figure 2 As shown in , when two ORAMs with periodic refresh and instant refresh are converted, such as RingORAM and PathORAM. Since instant refresh only accesses real blocks, the server can immediately detect that the remaining blocks are virtual blocks after the access. If these virtual blocks are subsequently accessed by periodic refresh, leakage will occur. This leakage will still occur even if the metadata of the virtual blocks is inherited. Therefore, it is necessary to refresh the buckets accessed by instant refresh before periodic refresh access.

[0062] S30. In response to the user's pre-access message, refresh the first data bucket and the metadata of the first data bucket in the currently accessed ORAM client based on the refresh algorithm to obtain the second data bucket and the metadata of the second data bucket. In response to the user's access request message, obtain the accessed data stored in the ORAM server, refresh the second data bucket and the metadata of the second data bucket based on the refresh algorithm to obtain the third data bucket and the metadata of the third data bucket.

[0063] In order to prevent the above leakage and realize efficient ORAM service conversion, that is, conversion between different ORAMs, the present invention designs a cross-ORAM data structure RecordMap for recording the access status of data buckets. Each data bucket is mapped to a pair of metadata (cnt, lim), where cnt represents the number of accesses to the current bucket, and lim represents the maximum number of safe accesses to the current bucket. Whenever the number of accesses to the bucket reaches the maximum number of safe accesses, the data bucket needs to be refreshed in time. For this purpose, the present invention designs a corresponding block refresh method EvictRecord.

[0064] In an embodiment of the present invention, step S30 may include the following execution process:

[0065] S31, based on the user's pre-access message, obtaining a refresh mode of the currently accessed ORAM, wherein the refresh mode includes a periodic refresh mode and an immediate refresh mode;

[0066] S32, if the currently accessed ORAM is in periodic refresh mode, traverse the data buckets in the ORAM, determine the data buckets that meet the refresh conditions, and set the metadata of the data buckets that meet the conditions as the initial value. The expression of the initial value of the metadata of the data bucket is;

[0067] RecordMap[bid]=(0,lim b )

[0068] Among them, lim b Indicates the maximum number of accesses to a data bucket in the base ORAM client, base ORAM indicates RingORAM, bid indicates a data bucket in the ORAM client, and RecordMap indicates metadata of a data bucket in the ORAM client;

[0069] S33, if the currently accessed ORAM is in the instant refresh mode, no operation is performed;

[0070] S34. Traverse the data buckets in ORAM again, and update the metadata values ​​of all data buckets that meet the refresh conditions from (cnt, lim) to (cnt+1, min(lim, lim′)), and obtain the metadata of the second data bucket and the second data bucket, where lim′ represents the maximum number of access times of the data bucket before refresh.

[0071] In an embodiment of the present invention, step S30 may also include the following execution process:

[0072] S35, traverse all second data buckets, if the currently accessed ORAM is in periodic refresh mode, reset the metadata of all second data buckets to initial values, if the currently accessed ORAM is in immediate refresh mode, reset the metadata of the second data bucket to (1, 1), and obtain the second data bucket after the metadata is reset;

[0073] S36, traverse the second data buckets after resetting the metadata again, if the currently accessed ORAM is in periodic refresh mode, reset each second data bucket after resetting the metadata to the initial value again, if the currently accessed ORAM is in immediate access mode, do not do anything.

[0074] It should be noted that when OST performs ORAM service conversion, V-ORAM will inherit the RecordMap of each ORAM and the metadata of the virtual block, regardless of whether the current ORAM is refreshed immediately or periodically.

[0075] In an embodiment of the present invention, before step S20, the variable adaptive oblivious storage method for dynamic workloads may further include the following specific execution process:

[0076] S02, in response to first configuration information input by a user, determining a server-side monetary cost based on a planner, wherein the server-side monetary cost includes a communication cost, a server-side computing cost, and a server-side storage cost, and the first configuration information is a client workload sequence;

[0077] S03, determining the server-side parameters of the variable ORAM based on the server-side monetary cost;

[0078] The planner of the present invention uses a given data set, client workload sequence, minimum throughput (X sys ) and the maximum delay (L sys ) to estimate the optimal ORAM parameters, server configuration, and ORAM service order. The planner's assumptions about ORAM are simplified as follows: server configuration and monetary cost, that is, the monetary cost of the server includes communication (traffic billing or bandwidth billing), computing (CPU core number n cpu ) and storage (data volume). Assuming network and hardware delays (L net and L IO ) is constant, the processing speed of a single core (P cpu ) are also constant and averaged based on the specific workload. Download and upload speeds (P down and P up) is also constant. The client overhead is determined based on the price of the cloud server. It should be noted that the server parameters of variable ORAM may include data volume, number of CPU cores n cpu , L net , L IO , P down and P up .

[0079] S04, determining a first cost model and a second cost model of the client based on the server parameters of the variable ORAM;

[0080] S05, determining the minimum monetary cost of each ORAM client based on the first cost model and the second cost model of the client;

[0081] Specifically, step S05 may include the following execution process:

[0082] S051. Determine a first overhead model based on a server parameter and a data capacity parameter of the variable ORAM, wherein the expression of the first overhead model is:

[0083]

[0084] Among them, op represents the client operation, which can represent the query operation of a certain ORAM or the conversion operation between any two ORAMs. Lop represents the client operation delay, C cpu represents the computing capacity of the client, C down Indicates the download capacity of the client, C up Indicates the upload capacity of the client, P cpu Indicates the number of CPU cores on the server, n cpu Indicates the processing speed of a single CPU core on the server, P down Indicates the download speed of the server, P up Indicates the upload speed of the server, L net Indicates the network delay of the server, L IO Indicates the hardware delay of the server;

[0085] S052. Based on the preset batch processing queue size and the delay of the client, determine the expression of the second overhead model as:

[0086]

[0087] Among them, X op Indicates the throughput of the client, and BatchSize indicates the batch queue size of the client.

[0088] S06. Determine the service order of each ORAM based on the minimum monetary cost.

[0089] Specifically, after step S06, the variable adaptive oblivious storage method for dynamic workloads may further include the following execution process:

[0090] S07. In response to the second configuration information input by the user, determine the parameters of each ORAM based on the planner.

[0091] Specifically, step S06 may include the following execution process:

[0092] S071, obtaining the size of the data bucket of each ORAM, the size of the data block in the data bucket, and the size of the data set to be stored;

[0093] S072: Determine a first bandwidth expansion parameter and a second bandwidth expansion parameter of each ORAM parameter based on the size of the data bucket, the size of the data block, and the size of the data set to be stored.

[0094] Wherein, step S072 may include the following execution process:

[0095] S071, determining the relationship between the data volume of the data set to be stored and the data volume of the data block;

[0096] S0722. If the data volume of the data set to be stored is smaller than the data volume of the data block, a first bandwidth expansion parameter of each ORAM is calculated using a first formula, where the first formula is:

[0097]

[0098] Wherein, Z represents the size of the data bucket, B represents the size of the data block, F represents the size of the data set to be stored in each ORAM, M represents the total size of the data set to be stored, and X represents the first bandwidth expansion parameter;

[0099] S073. If the data volume of the data set to be stored is greater than the data volume of the data block, a second bandwidth expansion parameter of each ORAM is calculated using a second formula, where the second formula is:

[0100]

[0101] Wherein, Z represents the size of the data bucket, B represents the size of the data block, M represents the total size of the data set to be stored, and represents the second bandwidth expansion parameter.

[0102] Specifically, the planner mainly selects the bucket size Z and the data size B, while other parameters follow the security constraints of ORAM. We assume that Z and B are determined by the client before deployment and remain unchanged throughout the V-ORAM process. For different file sizes F in practice, for example, image files at the MB level and text files at the KB level. Different file sizes require opposite parameter settings. Taking the commonly used indicator bandwidth expansion (the ratio of the amount of data transmitted to the request block size) of PathORAM as an example, for a data set with a total data volume of M, when F<B, the expansion is It decreases as B increases; when F>B, it expands to It increases with B. Therefore, there is an optimal parameter setting according to the client file size distribution and ORAM sequence.

[0103] In summary, the present invention is a variable adaptive oblivious storage method for dynamic workloads. First, a variable ORAM framework V-ORAM is designed to abstract the specific operations of ORAM. Secondly, the OST conversion protocol is given, and the specific conversion process is given by taking PathORAM, RingORAM and ConcurORAM as examples. The conversion overhead of OST is at a constant level, which is only related to the system security parameters and the parameters selected by ORAM itself, but has nothing to do with the number of data blocks in ORAM. In addition, the present invention points out the two types of leakage in the conversion process, and designs RecordMap and EvictRecord to generate only O(1) additional overhead to the original ORAM while ensuring security. Finally, in order to assist the client to make better decisions, the present invention provides a planner to assist the client in making decisions in various issues such as V-ORAM deployment, conversion and parameter selection. V-ORAM has been formally proven to ensure the confidentiality and obliviousness of the original ORAM data, and no data leakage will occur during the conversion process.

[0104] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A variable adaptive oblivious storage method for dynamic workloads, characterized in that: A client of a variable ORAM is applied, the client having a planner, the method comprising: In response to first configuration information input by a user, determining a service order of each ORAM based on the planner, wherein the ORAM stands for Oblivious Memory, each of the ORAMs includes RingORAM, PathORAM, and ConcurORAM, each of the ORAMs has a data bucket and metadata of the data bucket, and the metadata of the data bucket is determined based on a maximum access count and a current access count of the data bucket; In response to the service conversion request message of the user, using the RingORAM as an intermediary, converting between ConcurORAM services and / or PathORAM services; In response to a pre-access message from a user, a first data bucket and metadata of the first data bucket in the currently accessed ORAM client are refreshed based on a refresh algorithm to obtain a second data bucket and metadata of the second data bucket; in response to an access request message from the user, accessed data stored in the ORAM server is obtained, the second data bucket and metadata of the second data bucket are refreshed based on the refresh algorithm to obtain a third data bucket and metadata of the third data bucket.

2. The variable adaptive oblivious storage method for dynamic workloads according to claim 1, characterized in that: Before performing conversion between ConcurORAM and / or PathORAM by using RingORAM as an intermediary in response to the service conversion request message of the user, the method further includes: An encrypted database is generated in response to the content information input by the user, and the encrypted database is sent to the server of the variable ORAM by using the planner.

3. The variable adaptive oblivious storage method for dynamic workloads according to claim 1, characterized in that: Each of the ORAMs has a client, a server, a server API, and a client API. In response to the user's service conversion request message, the RingORAM is used as an intermediary to complete the conversion between ConcurORAM services and / or PathORAM services, including: Encrypting and processing the local cache data, and filling the local cache data with virtual blocks to obtain first synchronization metadata, uploading the first synchronization metadata to the ConcurORAM server, and using the ConcurORAM client API to download the first synchronization metadata and the second synchronization metadata, and restoring the local cache data based on the first synchronization metadata and the second synchronization metadata, respectively switching the ConcurORAM server API, the client API and the data structure to the RingORAM server API, the client API and the metadata of the data bucket, and completing the conversion of the ConcurORAM service to the RingORAM service; The server API, the client API and the data structure of the RingORAM are respectively switched to the server API, the client API and the metadata of the data bucket of the PathORAM, thereby completing the conversion of the RingORAM service to the PathORAM service.

4. The variable adaptive oblivious storage method for dynamic workloads according to claim 1, characterized in that: The method of responding to the user's pre-access message and refreshing the first data bucket and metadata of the first data bucket in the currently accessed ORAM client based on the refresh algorithm to obtain the second data bucket and metadata of the second data bucket includes: Based on the pre-access message of the user, obtaining a refresh mode of the currently accessed ORAM, wherein the refresh mode includes a periodic refresh mode and an immediate refresh mode; If the previously accessed ORAM is in periodic refresh mode, traverse the data buckets in the ORAM, determine the data buckets that meet the refresh conditions, and set the metadata of the data buckets that meet the conditions as initial values. The expression of the initial value of the metadata of the data bucket is: RecordMap[bid]=(0,lim b ) Among them, lim b represents the maximum number of accesses to a data bucket in a base ORAM client, where the base ORAM is RingORAM, bid represents a data bucket in the ORAM client, and RecordMap represents metadata of the data bucket in the ORAM client; If the currently accessed ORAM is in immediate refresh mode, no operation is performed; Traverse the data buckets in the ORAM again, and update the metadata values ​​of all the data buckets that meet the refresh conditions from (cnt, limm) to (cnt+1, mmin(lim, limm′)), and obtain the second data bucket and the metadata of the second data bucket, where limm′ represents the maximum number of accesses to the data bucket before refresh.

5. The variable adaptive oblivious storage method for dynamic workloads according to claim 4, characterized in that: The step of obtaining the accessed data stored in the ORAM server in response to the access request message of the user, refreshing the second data bucket and the metadata of the second data bucket based on the refresh algorithm, and obtaining the third data bucket and the metadata of the third data bucket includes: Traverse all the second data buckets. If the currently accessed ORAM is in periodic refresh mode, reset the metadata of all the second data buckets to the initial value. If the currently accessed ORAM is in immediate refresh mode, reset the metadata of the second data bucket to (1, 1), and obtain the second data bucket after resetting the metadata. The second data buckets after the metadata is reset are traversed again. If the currently accessed ORAM is in periodic refresh mode, each second data bucket after the metadata is reset is reset to the initial value again. If the currently accessed ORAM is in immediate access mode, no operation is performed.

6. The variable adaptive oblivious storage method for dynamic workloads according to claim 1, characterized in that: The step of determining the service order of each ORAM based on the planner in response to the first configuration information input by the user comprises: In response to first configuration information input by the user, determining a server-side monetary cost based on the planner, wherein the server-side monetary cost includes a communication cost, a server-side computing cost, and a server-side storage cost, and the first configuration information is a client workload order; Determining a server-side parameter of the variable ORAM based on the server-side monetary cost; Determine a first cost model and a second cost model of the client based on the server parameters of the variable ORAM; Determining a minimum monetary cost for each client of the ORAM based on a first cost model and a second cost model of the client; A service order of each of the ORAMs is determined based on the minimum monetary cost.

7. The variable adaptive oblivious storage method for dynamic workloads according to claim 6, characterized in that: The determining of the first cost model and the second cost model of the client based on the server parameters of the variable ORAM includes: The first overhead model is determined based on the server parameters and data capacity parameters of the variable ORAM, wherein the expression of the first overhead model is: Among them, op represents the client operation, which can represent the query operation of a certain ORAM or the conversion operation between any two ORAMs. Lop represents the client operation delay, C cpu represents the computing capacity of the client, C down Indicates the download capacity of the client, C up Indicates the client's upload capacity n cpu , indicates the number of CPU cores on the server, P cpu Indicates the processing speed of a single CPU core on the server, P down Indicates the download speed of the server, P up Indicates the upload speed of the server, L net Indicates the network delay of the server, L IO Indicates the hardware delay of the server; Based on the preset batch queue size and the delay of the client, the expression of the second overhead model is determined as: Among them, X op Indicates the throughput of the client, and BatchSize indicates the batch queue size of the client.

8. The variable adaptive oblivious storage method for dynamic workloads according to claim 6, characterized in that: After determining the service order of each ORAM based on the server currency and the client cost, the method further includes: In response to the second configuration information input by the user, parameters of each of the ORAMs are determined based on the planner.

9. The variable adaptive oblivious storage method for dynamic workloads according to claim 8, characterized in that: The step of determining the parameters of each of the ORAMs based on the planner in response to the second configuration information input by the user comprises: Obtain the size of each ORAM data bucket, the size of the data block in the data bucket, and the size of the data set to be stored; A first bandwidth expansion parameter and a second bandwidth expansion parameter of each ORAM parameter are determined based on the size of the data bucket, the size of the data block, and the size of the data set to be stored.

10. The variable adaptive oblivious storage method for dynamic workloads according to claim 9, characterized in that: The first bandwidth expansion parameter and the second bandwidth expansion parameter of each ORAM parameter are determined based on the size of the data bucket, the size of the data block and the size of the data set to be stored, including: Determine the relationship between the data volume of the to-be-stored data set and the data volume of the data block; If the data volume of the data set to be stored is smaller than the data volume of the data block, a first bandwidth expansion parameter of each ORAM is calculated using a first formula, where the first formula is: Wherein, Z represents the size of the data bucket, B represents the size of the data block, F represents the size of the data set to be stored in each ORAM, M represents the total size of the data set to be stored, and X represents the first bandwidth expansion parameter; If the data volume of the data set to be stored is greater than the data volume of the data block, a second bandwidth expansion parameter of each ORAM is calculated using a second formula, where the second formula is: Wherein, Z represents the size of the data bucket, B represents the size of the data block, M represents the total size of the data set to be stored, and Y represents the second bandwidth expansion parameter.

Images