Intellectual property database searchable encryption method supporting dynamic updating

Abstract

The invention discloses an intellectual property database searchable encryption method supporting dynamic updating, and aims to solve the problems of high index reconstruction overhead and low updating efficiency in a data addition scene in an existing scheme. The core of the method is to construct a layered and merged dynamic index structure: for newly added data, a client generates an independent lightweight incremental index fragment (L0) and uploads the fragment, so that low-delay updating is realized; and when the incremental index fragments are accumulated to a threshold value, triggering a background hierarchical merging mechanism to merge a plurality of low-level index fragments into high-level index fragments so as to maintain the retrieval efficiency. By storing the index entries in the confusion bucket located by trap door hash, the association between the index entries is cut off, and the access mode privacy is enhanced. In the searching stage, the server side performs intelligent filtering through a bloom filter, the minimum target ciphertext set is determined, and network transmission and client side decryption expenses are greatly reduced. According to the method, the data confidentiality and the retrieval privacy are ensured, and meanwhile, the system performance in a dynamic environment is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of computer data security and database technology, specifically relating to a method for keyword search on encrypted data, and more particularly to a method for constructing a searchable encrypted dynamic index for keywords that supports efficient updates for highly dynamic application scenarios such as intellectual property databases. Background Technology

[0002] Intellectual property databases (such as patents and copyrights) typically contain massive amounts of highly structured text data, which possesses immense commercial and strategic value. However, due to the extremely high commercial value and sensitivity of intellectual property data, storing it directly in plaintext in the cloud poses significant security risks, including external attacks and internal threats to cloud service providers.

[0003] To address this issue in the future, data encryption has become a necessary security measure. However, traditional encryption methods result in the loss of data searchability, preventing users from performing keyword searches on encrypted data. Searchable encryption technology has emerged to address this problem, allowing keyword searches to be performed on data while maintaining its encryption.

[0004] Most existing searchable encryption schemes focus on static datasets, assuming that the data remains unchanged after indexing. Intellectual property databases are highly dynamic systems, with new patent applications, trademark registrations, and other information frequently appended. For static schemes, each new record may require rebuilding the entire inverted index of the keywords, a process involving large-scale decryption, appending, re-encryption, and uploading operations, resulting in significant computational overhead and time latency, failing to meet the efficiency requirements of practical applications. Therefore, designing a searchable encryption scheme that guarantees both data confidentiality and query privacy while supporting efficient, low-cost dynamic updates is a key technical challenge in this field. Furthermore, the "dynamic" aspect proposed in this invention does not refer to changes in the encryption algorithm itself, but rather to the dynamic scalability of the index structure. Summary of the Invention

[0005] The present invention aims to overcome the shortcomings of existing searchable encryption schemes in the background art when processing dynamic data, and provides a method for constructing a keyword searchable encrypted dynamic index for intellectual property databases with high update efficiency, low retrieval latency, and high security.

[0006] To achieve the above objectives, this invention provides a dynamic indexing method employing a hierarchical merging strategy. This method splits the complete inverted index of a keyword into a stable, large master state index (L2) and multiple temporary, small append state indices (L0). Specifically, the technical solution of this invention includes:

[0007] 1. Lightweight Appendage: When a new document ID needs to be associated with a keyword, the system does not touch the existing massive main index. Instead, it creates a completely new, independent L0-level index entry for this single update operation. This index only contains the encrypted value of the new ID and a Bloom filter, so its generation and uploading process is very fast.

[0008] 2. Hierarchical and Stateful: By introducing the concepts of "Level" and "State," the index of each keyword consists of multiple independent parts at different levels. The client maintains a local state database, recording which levels, states, and corresponding trapdoors each keyword possesses.

[0009] 3. Obfuscation of Storage and Access: To prevent the server from inferring user search patterns by associating index items in different states, this invention introduces an obfuscation bucket mechanism. When an index item is uploaded, it is not stored directly using its trapdoor as the storage key, but rather in a bucket determined by the trapdoor hash value. This causes different index items for the same keyword to be scattered and stored in seemingly unrelated locations, making it impossible for the server to directly associate them. During a search, the server also needs to locate the bucket through the hash before matching, thus effectively obfuscating the actual access pattern.

[0010] 4. Asynchronous Background Merging: When the number of L0-level index entries for a keyword reaches a certain level (e.g., 10), to avoid index fragmentation affecting query efficiency, the system will automatically trigger a merging task in the background. This task downloads, decrypts, and merges these L0 index entries, then re-encrypts them to generate a larger, higher-level L1 index entry, and deletes the original L0 entries. This process is seamless for front-end update and query operations.

[0011] 5. Server-side intelligent filtering: When performing multi-keyword joint queries, the client sends all levels of trapdoors for all relevant keywords to the server. After the server uses these trapdoors to find all index segments, it does not directly manipulate the large encrypted list. Instead, it first compares the Bloom filters of each segment and intelligently determines which keyword has the smallest total document list length. The server only returns the encrypted list of this smallest set, along with the Bloom filters of all other segments. This significantly reduces network overhead and the client's decryption burden.

[0012] Compared with existing technologies, the beneficial effects of this method are reflected in:

[0013] 1. Extremely high update efficiency: Through the design of L0 append state, new data only needs to be encrypted and uploaded once, realizing an update with a near O(1) complexity, which meets the high dynamism requirements of intellectual property database.

[0014] 2. Low query latency: The background merging mechanism effectively controls the number of index fragments, ensuring that the number of index segments to be processed during a query remains within a controllable range. Simultaneously, the server-side intelligent filtering mechanism significantly reduces the amount of response data, accelerating subsequent client-side processing.

[0015] 3. Robust Security: This solution inherits the advantages of searchable encryption, ensuring the confidentiality of data content and search keywords to the server. The layered and stateful design also obfuscates access patterns to a certain extent, enhancing security.

[0016] 4. Excellent scalability: The layered merging design enables the system to smoothly handle the continuous growth of data scale. Attached Figure Description

[0017] Figure 1 This is a system overall architecture diagram according to an embodiment of the present invention;

[0018] Figure 2 This is a flowchart of the primary index initialization and construction in an embodiment of the present invention;

[0019] Figure 3 This is a flowchart of incremental data update in an embodiment of the present invention;

[0020] Figure 4 This is a flowchart of index hierarchical merging in an embodiment of the present invention;

[0021] Figure 5 This is a flowchart of a multi-keyword encrypted search in an embodiment of the present invention. Detailed Implementation

[0022] like Figure 1 As shown, the system architecture proposed in this invention mainly consists of two parts: a client and a server.

[0023] The client is a trusted entity, and its main modules include:

[0024] 1. Index building module: Responsible for processing initial and new data, performing encryption operations and generating L2 primary index or L0 incremental index fragments, and uploading them to the server.

[0025] 2. Local State Database: This is a lightweight local storage used to record the current "hierarchical state" of each keyword (i.e., which L0 fragments it contains, and whether there are L1 merged fragments), and to track the count of incremental updates.

[0026] 3. Search and decryption module: Responsible for generating search trapdoors and decrypting and verifying candidate results returned by the server using local Bloom filtering.

[0027] 4. Backend Merge Module: Independent of the frontend operation, when the local status database detects that Count > k, the merge logic is triggered asynchronously to download the old fragment, generate a new L1 index, and upload it.

[0028] The server is a semi-trusted entity, and its main modules include:

[0029] 1. Query processing module: Receives client trapdoors, reads data from cloud storage, and executes a "smart filtering" strategy based on Bloom filters.

[0030] 2. Cloud storage: Stores encrypted index data, including L2 primary index, L1 merged index and L0 index fragments.

[0031] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0032] I. Index Building (Initial Primary Index Building)

[0033] like Figure 2 As shown, when the system is deployed for the first time or when existing data is initialized, the following steps are performed:

[0034] 1. Preprocess all records in the intellectual property database, construct a temporary plaintext inverted index, and map each keyword (W) to a list of all document IDs associated with it (L).

[0035] 2. Iterate through each keyword W and its list L. Assign an initial, top-level "main state" to the keyword, for example, st = "L2_main_0".

[0036] 3. Using a pseudo-random function (PRF), combined with the master key (MK), keyword W, and state st, generate the main trapdoor Tw = prf_f(MK, W+st) and the master key K_W = prf_g(MK, W+st).

[0037] 4. Encrypt the entire list L using the master key K_W with a symmetric encryption algorithm to obtain C_L.

[0038] 5. Construct a Bloom filter F_L for the entire list L.

[0039] 6. Combine Tw, C_L, and F_L into the primary index entry Entry.

[0040] 7. Finally, record the following in the client's local state database DB: DB["W"] = [{"level":2,"state":"L2_main_0", "trapdoor":Tw}].

[0041] II. Dynamic Updates (Triggered by Append-to-Index Building and Merging)

[0042] This step involves building the incremental index. Unlike re-encrypting the entire inverted index, this method uses a dynamic indexing strategy, generating independent index segments (L0) for each new data entry. For example... Figure 3 As shown, when the system receives a new update request (keyword W, new document ID is new_ID):

[0043] 1. The client generates a new L0 state for this update, such as st_new_0, and derives the corresponding append trapdoor Tw_new and append key K_W_new.

[0044] 2. Encrypt only the new ID, C_L_new = Encrypt(K_W_new, [new_ID]), and build a tiny Bloom filter F_L_new only for this new ID.

[0045] 3. Combine them into a lightweight append index entry Entry_new = (Tw_new, C_L_new, F_L_new).

[0046] 4. To obfuscate access patterns and prevent the server from associating multiple updates belonging to the same keyword, the client does not directly upload Tw_new as the address. Instead, it calculates its hash value to obtain the bucket address: Bucket_ID = Hash(Tw_new). 5. Then, the client uploads the entire index entry Entry_new to the server, specifying that it be stored in the bucket identified by Bucket_ID. Due to the avalanche effect of hash functions, even two trapdoors with the same keyword and similar states will have completely different and unpredictable bucket addresses.

[0047] 6. Append this new L0 status information to the local DB["W"]: .append({"level":0, ...}).

[0048] 7. After the update is complete, check if the number of entries with level 0 in DB["W"] is greater than the threshold k. If so, send a merge command asynchronously to start the background "hierarchical merge" task. The choice of threshold k is a performance trade-off parameter; a smaller k value will result in more frequent merges, increasing background computational overhead, but maintaining high search efficiency; a larger k value has the opposite effect. In practical applications, it can be adjusted according to the system's update frequency and performance requirements.

[0049] III. Dynamic Updates (Layered Merging)

[0050] like Figure 4 As shown, this background task is responsible for controlling index fragmentation:

[0051] 1. After the task starts, retrieve all entries with level 0 for the keyword W from the database, along with their trapdoor list.

[0052] 2. Traverse the trapdoor list and download all corresponding L0 index entries from the cloud server.

[0053] 3. On the client's local machine, each L0 item is decrypted using its own key to obtain multiple list fragments containing a small number of IDs.

[0054] 4. Union all these list fragments into a larger list L_merged.

[0055] 5. Create a new L1 level state (such as st_new_1) for L_merged, and generate a new L1 trapdoor, key, encryption list and Bloom filter for it, forming an L1 index entry Entry_L1.

[0056] 6. Upload Entry_L1 to the server and update the local database. At the same time, delete all merged L0 entries and add the newly generated L1 entries.

[0057] 7. Finally, a delete command containing trapdoors for all L0 index entries to be deleted is sent to the server. Based on these trapdoors, the server locates these old index entries using the same "hash-bucket-match" process as during the search and precisely removes them from the storage bucket, thereby cleaning up redundant data on the server. The L1 to L2 merging process is similar, but it involves larger amounts of data and is triggered less frequently.

[0058] IV. Search Process

[0059] like Figure 5 As shown, when a user executes a multi-keyword query, such as "W1 AND W2":

[0060] 1. The client queries the local database to obtain a list of trapdoors at all levels associated with W1 and W2, for example, Traps_W1 = [Tw_L2_A, Tw_L0_A, Tw_L0_B] and Traps_W2 = [Tw_L2_B, Tw_L1_B].

[0061] 2. The client sends these trapdoor lists to the server as search requests.

[0062] 3. After receiving the trapdoor list, the server performs the following operations for each trapdoor T in the list:

[0063] a. Calculate the bucket address: Bucket_ID = Hash(T).

[0064] b. Retrieve the entire bucket corresponding to the Bucket_ID from the storage system. A bucket may contain multiple unrelated index entries due to hash collisions.

[0065] c. Traverse within the bucket and find the index item whose trapdoor portion completely matches T.

[0066] Through the above steps, the server finally finds the set of index entries (Entry_L2_A, Entry_L0_A,...) corresponding to all request trapdoors.

[0067] 4. The server performs intelligent filtering. It doesn't decrypt any content; instead, it determines which keyword's associated document ID list is likely the shortest by comparing the Bloom filter sizes of the index entries corresponding to different keywords. For example, it compares the total number of documents represented by Entry_L2_A, Entry_L0_A, and Entry_L0_B with the total number of documents represented by Entry_L2_B and Entry_L1_B. If the total number of documents in W1 is significantly greater than that in W2, then the list set corresponding to W2 is considered the "minimum set".

[0068] 5. The server returns all encrypted lists of W2 [C_L_L2_B, C_L_L1_B] as C_L_L1_list, and returns all Bloom filters of W1 [F_L_L2_A, F_L_L0_A, F_L_L0_B] as OtherFilter.

[0069] 6. After receiving the response, the client only needs to decrypt and merge C_L_min_list to obtain L_W2_merged. Then, it uses each Bloom filter in OtherFilter to examine each ID in L_W2_merged. Only the IDs that pass the examination of all filters are the final intersection result L_final.

[0070] In summary, the embodiments of the present invention, through ingenious layered design and dynamic merging strategy, successfully construct a secure, efficient, dynamically updated searchable encrypted indexing method that can well meet the application needs of databases in key fields such as intellectual property.

Claims

1. A searchable encryption method for a database that supports dynamic updates, characterized by incremental updates. The index building phase and the merging task triggering and execution phase, wherein the incremental index building phase includes: when the client receives an update request containing a keyword (W) and a new document ID, it generates an independent incremental state (L0) for this update; based on the keyword (W) and the incremental state (L0), it derives an incremental trapdoor (Tw_new) and an incremental key (K_W_new); it uses the incremental key (K_W_new) to encrypt the new document ID to generate an encrypted document ID (C_L_new); and it constructs an incremental Bloom filter (F_L_new) for the new document ID; it combines the incremental trapdoor (Tw_new), the encrypted document ID (C_L_new), and the incremental Bloom filter (F_L_new) into an incremental index fragment, characterized in that the client calculates the hash value of the incremental trapdoor (Tw_new) to generate a storage address (Bucket_ID), and uploads the incremental index fragment to the storage bucket identified by the storage address on the server; at the same time, it records the association information between the keyword (W) and the incremental state (L0) in the client's local state database. The merging task triggering and execution phase includes: the client monitors the local state database, and when the number of incremental states associated with a certain keyword exceeds a preset threshold, a hierarchical merging task is triggered; this task downloads all corresponding incremental index fragments from the server, decrypts and merges the document ID list locally on the client, generates a new high-level state (L1) and the corresponding merged index fragment; the merged index fragment is uploaded to the server, the local state database is updated, and the server is instructed to delete the old incremental index fragments.

2. The method as described in claim 1, characterized in that, The method also includes an initial master index construction step: during system initialization, a top-level "master state" (L2) is generated for each keyword (W) and its complete initial document ID list (L), and a master trapdoor and master key are derived based on this; the entire document ID list (L) is encrypted using the master key, and a Bloom filter is built for it, which is then combined into a master index item and uploaded to the server, and the master state information is recorded in the local database.

3. The method as described in claim 1, characterized in that, The method further includes a secure search step: the client retrieves all trapdoors at all levels associated with the user's input query keywords from the local state database, including main state trapdoors and all unmerged incremental state trapdoors; the client then sends all trapdoors to the server. The server performs intelligent filtering: for each trapdoor sent by the client, the server first calculates its hash value to obtain the corresponding storage address, and then obtains the entire storage bucket identified by that address. Then, matching is performed within the storage bucket to find index entries that are completely consistent with the current trapdoor. After finding all index entries corresponding to all trapdoors, the minimum set containing the fewest document IDs is determined based on the length of the document list associated with each index entry. After receiving the response, the client only needs to decrypt and merge the encrypted list of the minimum set, and then use the other received Bloom filters to perform intersection filtering on the merged document ID list locally to obtain the final accurate search results.

4. The method as described in claim 1, characterized in that, The incremental trapdoor (Tw_new) and incremental key (K_W_new) are generated by using a random function (PRF) with the master key (MK), keyword (W), and a string representing the incremental state (L0) as input.

Images