Context-enhanced semantic searchable encryption method and system

Abstract

The invention discloses a context-enhanced semantic searchable encryption method and a context-enhanced semantic searchable encryption system. The method comprises the following steps of: 1) training a selected model for data by using a labeled query-document to obtain an optimal model, and deploying the optimal model at a client; 2) the client generates a secret key, generates a document vector representation hd for a to-be-uploaded document by using the optimal model, then encrypts the hd by using the secret key to obtain an encrypted document vector, and uploads the encrypted document vector to the server; 3) the server side stores the document vector and constructs a corresponding index I; 4) the client generates a query vector representation hq for the input query through the optimal model, and then encrypts the hq by using the key to obtain a final encrypted query trap door Tq and uploads the query trap door Tq to the server; and 5) the server side calculates a similarity score between the query trapdoor Tq and the index I, and first k results with the highest score are selected and returned to the client side. According to the method, the search accuracy is ensured, and the data security is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of information security, and relates to a context-enhanced semantic searchable encryption method and system. Background Art

[0002] Cloud storage technology has been widely adopted as a mainstream technology for data storage due to its significant advantages of efficiency and convenience. As a result, more and more data owners choose to outsource their data to cloud servers. However, cloud servers tend to be "honest but curious", performing their retrieval duties correctly but also trying to obtain private information by inferring and analyzing the data in their storage (including indexes) and the message streams received during the protocol. In order to protect the privacy security of data, data owners should encrypt their data before outsourcing it to the cloud. This in turn makes data utilization such as searching encrypted data a challenging problem.

[0003] In order to achieve secure search of encrypted data, Song et al. first proposed the concept of searchable encryption. Since then, searchable encryption has attracted much attention, and many searchable encryption schemes have been proposed. Searchable encryption schemes usually include five basic steps: document feature extraction, searchable index construction, query trapdoor generation, trapdoor-based index search, and search result return. Traditional searchable encryption schemes only support exact matching of query keywords, which will result in incomplete search results or fail to fully meet the user's query request. The same meaning can be expressed in different words, such as "Peking University" and "Peking University". If the user's query keyword is "Peking University", the document containing "Peking University" cannot be retrieved, resulting in incomplete search results. This limitation stems from the inability of traditional searchable encryption schemes to effectively discover the semantic relationship between words.

[0004] Therefore, researchers have proposed semantic searchable encryption schemes that support more flexible queries by considering semantic information to meet different user needs. Generally speaking, semantic searchable encryption schemes generate vectors with semantic information in the feature extraction stage. These vectors are then encrypted using an inner product encryption algorithm that supports semantic search. The most relevant documents are finally returned by calculating the similarity scores between the query trapdoor and the encrypted document index and sorting these scores. Early semantic searchable encryption schemes used vector space models to obtain query and document vectors and added additional semantic information by expanding the query terms, which alleviated the problem of vocabulary mismatch to a certain extent. These semantic searchable encryption schemes can be roughly divided into synonym-based schemes, mutual information model-based schemes, and concept hierarchy-based schemes. However, these vector space model-based schemes only consider the frequency of occurrence of each word, while ignoring the relationship between words and contextual information, which will lead to suboptimal search results.

[0005] To explore contextual information, recent semantic searchable encryption schemes utilize common word embedding models such as Word2Vec, Glove, and FastText to generate word embeddings with contextual information by training documents. Although there have been many advances in semantic searchable encryption, there are still two key issues that limit accuracy and efficiency. First, in current semantic searchable encryption schemes, the feature extraction of queries and documents is not sufficient, and the generated vector representation does not consider the semantic relevance between queries and documents. Specifically, in the semantic searchable encryption task, the retrieval process relies on calculating the distance between queries and documents. Minimizing the distance between query and related document embeddings plays an important role in accurately identifying closely related results. Therefore, how to effectively capture this relevance and optimize the vector representation of queries and documents to improve accuracy remains a major challenge.

[0006] Second, most existing searchable encryption schemes use inverted indexes, clustering methods, or tree index structures, which are limited when processing large-scale datasets. As the size of the dataset grows to millions or billions of numbers, the computational complexity of the inverted index or clustering algorithm grows significantly, and the tree structure becomes more complex, which will increase storage overhead and reduce search efficiency. Therefore, how to scale up the secure search process to handle large-scale datasets while maintaining low latency response time and small memory cost remains a major challenge. Summary of the invention

[0007] In view of the shortcomings of the existing semantic searchable encryption methods analyzed above, the purpose of the present invention is to provide a context-enhanced semantic searchable encryption method and system, which can realize semantic search of encrypted data. The present invention uses three components to ensure accuracy, security and efficiency; first, a context-enhanced pre-trained model component is proposed to ensure search accuracy. Specifically, the component uses a Transformer-based pre-trained language model trained with a large number of query document pairs to obtain better query / document vector representation. On this basis, the component fine-tunes the model in a contrastive learning manner so that the query and related documents are close in distance in the embedding space, while the distance between the query and the irrelevant documents is far. Second, the scheme considers that under a more realistic threat model, the cloud server can collude with data users to collect information from documents and index structures; an optimized asymmetric scalar product preserving encryption (ASPE) algorithm component is proposed to ensure the security of the scheme. The component optimizes the asymmetric scalar product preserving encryption algorithm to make it suitable for this scheme, and uses it to encrypt query vectors and document vectors. Third, a secure ranked search component is proposed to achieve efficient search on large-scale datasets, which constructs an approximate nearest neighbor (ANN) index for encrypted document vectors and performs fast search.

[0008] The specific steps of the context-enhanced semantic searchable encryption method provided by the present invention are as follows:

[0009] Step 1. Model pre-training. Use labeled query-document pairs to train the selected model. Each training data contains a query-document pair and its corresponding label information. During the training process, the distance between the query and the relevant documents is minimized by calculating the contrast loss. The vector representation of the query and the document is optimized by contrastive learning, so that the distance between the query and the relevant documents is closer and the distance to the irrelevant documents is farther. When the model parameters converge and the retrieval effect on the test data is optimal, save the optimal model at this time.

[0010] Step 2. Build the trained optimal model on the client. The client first generates the key {S, M1, M2}. S is a randomly generated (m+1)-dimensional bit vector, M1 and M2 are reversible matrices of size (m+1)×(m+1), and m is the fixed vector dimension generated by the optimal model.

[0011] Step 3. The client generates an m-dimensional document vector representation h for the document to be uploaded using the optimal model d , and then use the generated key {S,M1,M2} to represent the document vector h d Encryption is performed. The encryption process is as follows: First, create a (m+1)-dimensional vector vector The first m dimensions are h d . Then the (m+1)th dimension is set to -0.5||h d || 2 , then the document vector The final expression is (h d ,-0.5||h d || 2 ). Then, to enhance security, we transform the document vector S into Each dimension of is randomly split into two random vectors Specifically, if the j-th dimension S[j] in vector S is 0, then the random vector of the split j-th dimension and a random vector j-th dimension is set to the same vector The jth dimension The same value. If S[j] is 1, then and are assigned random values ​​while ensuring that their sum is equal to Next, we use the reversible matrices M1 and M2 in the key to further and Encrypt and get and Finally, we concatenate the two split vectors into one to get the final encrypted document vector

[0012]

[0013] Step 4. The client sends the encrypted document vector Enc(h d ) is uploaded to the server, and the server will encode the document vector Enc(h d ) is stored in the open source vector database FAISS, and is the encrypted document vector Enc(h d ) constructs the corresponding ANN index I = Enc(h d ).

[0014] Step 5. The client generates an m-dimensional query vector representation h for the query to be searched through the optimal model q , and then use the generated key to query the vector representation h q Encryption is performed. The encryption process is as follows: First, create a (m+1)-dimensional vector vector The first m dimensions are h q Then the (m+1)th dimension is set to 1, and scaled by a random number r greater than 0, querying the vector The final expression is (rh q ,r). Then, to enhance security, we transform the query vector S into Each dimension of is randomly split into two random vectors Specifically, if S[j] is 0, then the random vector of the split The jth dimension and a random vector The jth dimension in are assigned random values ​​while ensuring that their sum is equal to If S[j] is 1, and is set to Next, we use the reversible matrices M1 and M2 in the key to further and Encrypt and get and Finally, we concatenate the two split vectors into one to obtain the final encrypted query trapdoor.

[0015] Step 6. Client uploads query trap door T q The server receives the query trap Tq Then, the query trap door T is calculated using the vector database q The similarity score between the encrypted document index I is used to perform the sorted search. The specific similarity score and similarity calculation formula are as follows:

[0016]

[0017] The server sorts the similarity scores in descending order and selects the top k results with the highest scores and returns them to the client.

[0018] The client uses the key {S, M1, M2} to decrypt each returned result.

[0019] The key points of the present invention include:

[0020] 1. The semantically searchable encryption method proposed in this method integrates the context-enhanced pre-trained model, encryption algorithm and secure ranking search into an end-to-end framework, which can efficiently generate encrypted query / document vectors and significantly improve the effect of ciphertext semantic retrieval.

[0021] 2. This method proposes a new context-enhanced pre-training model that can better learn the vector representation of each query and document and improve search accuracy.

[0022] 3. This method designs a new way to build document indexes, where each encrypted document vector corresponds to an ANN index, which can perform fast searches.

[0023] 4. This method considers a more realistic threat model and optimizes the ASPE encryption algorithm to ensure security.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. In view of the problem that the feature extraction of queries and documents in current semantic searchable encryption schemes is not sufficient and the generated vector representation does not take into account the semantic relevance between queries and documents, the present invention proposes a context-enhanced pre-trained model component that uses contrastive learning to narrow the difference between queries and related documents in vector representations, so that retrieval can return the most relevant documents, thereby ensuring the accuracy of the search.

[0026] 2. In view of the problem that the current searchable encryption methods using inverted index, clustering method, or tree index structure have high storage overhead and poor search efficiency when processing large-scale data sets, the present invention proposes a secure sorting search component. By building a vector database on the server side, the vector database is used to store encrypted document vectors and directly build an ANN index. Since the ANN index is directly built with encrypted document vectors, no additional time overhead is generated. The index memory is affected by the number of documents and the dimension of the encrypted document vector. The present invention generates a smaller encrypted vector dimension to reduce storage overhead, and the use of the vector database significantly improves the search efficiency.

[0027] 3. The current semantic searchable encryption scheme only uses the threat model of ciphertext-only attack. Once the server and the data user collude to carry out a chosen plaintext attack, the server can recover all the plaintext information. The present invention provides a stronger and more practical threat model, namely the chosen plaintext attack model, which can ensure the security of the scheme in this paper under the combination of the ASPE encryption algorithm and the scheme in this paper. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the process of the present invention.

[0029] Figure 2 It is a schematic diagram of the framework structure proposed by the method of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described in detail below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention but not to limit the scope of the present invention.

[0031] Figure 1 The overall flow chart of the present invention is shown in FIG. 1 . The system of the present invention includes two entities, the client and the server. The main functions of the client are to initialize system parameters, generate keys, perform model pre-training, generate query and document vectors using the trained optimal model, encrypt the document vectors and upload them to the server, generate query traps and submit query requests to the server. The server is responsible for saving the ciphertext documents uploaded by the client, storing the encrypted document vectors and building the ANN index, performing sorted retrieval and returning relevant retrieval results.

[0032] The functions of the client mainly include the following four aspects:

[0033] (1) Use annotated information retrieval datasets to pre-train the model and save the best model in the test set;

[0034] (2) Use the optimal model to generate the corresponding document vector for the document to be uploaded by the client. After the encryption key is generated, the key is used to encrypt the document vector and the document, and the encrypted document vector and ciphertext document are uploaded to the server;

[0035] (3) Generate a corresponding query vector for the query word using the optimal model, then encrypt it with a key and upload the generated query trapdoor to the server;

[0036] (4) Receive the query results returned by the server and decrypt the query results.

[0037] The functions of the server mainly include the following two aspects:

[0038] (1) After receiving the ciphertext document and encrypted document vector uploaded by the client, the server stores the document vector in the constructed vector database and constructs the corresponding ANN index for it;

[0039] (2) After receiving the query trapdoor uploaded by the client, the server uses the vector database to calculate the inner integral score between the query trapdoor and the document index and performs a sorted search, returning the top k search results to the client.

[0040] The specific implementation language of the present invention is Python programming language, and the index construction and search process rely on the open source vector database Faiss ANN library.

[0041] In order to better express the context-enhanced semantic searchable encryption scheme proposed in the present invention, the present invention is further described below in conjunction with specific implementation examples.

[0042] Step 1. Model pre-training. Divide the information retrieval dataset into two parts: training set and test set. Each data instance contains a query-document pair and its corresponding label. During the training process, the distance between the query and the relevant document is minimized by calculating the contrast loss. Continuously update the model parameters until the model converges and the effect in the test set is optimal. Save the optimal model at this time.

[0043] Step 2. Initialize the overall framework. Figure 2 It is the overall framework designed in the present invention, which includes a context-enhanced pre-trained model component, an encryption algorithm component, and a secure sorting search component. The context-enhanced pre-trained model component and the encryption algorithm component are built on the client side, and the secure sorting search component is on the server side.

[0044] In this embodiment, a document or query data passes through the context-enhanced pre-trained model component to obtain a 384-dimensional feature vector. In the encryption algorithm component, a vector passes through the encryption layer to obtain two 384-dimensional ciphertext vectors, which are then input into the splicing layer to be spliced ​​into a 768-dimensional ciphertext vector. After the server receives the 768-dimensional ciphertext vector, in the secure sorting search component, the ciphertext vector of the document is directly stored in the vector database and the corresponding ANN index is constructed; once the query ciphertext vector is received, the inner integral score between it and the document index is calculated and sorted and searched, and finally the top k search results are returned in descending order according to the inner integral score.

[0045] Step 3. The client generates a 384-dimensional document vector representation h for the document to be uploaded using the trained model. d , and then use the generated key to encrypt the document vector. The encryption process is as follows: First, create a 385-dimensional vector vector The first 384 dimensions are h d . Then set the 385th dimension to -0.5||h d || 2 , then the document vector The final expression is (h d ,-0.5||h d || 2 ). Then, based on the key S, the document vector Each dimension is randomly split into two random vectors Specifically, if S[j] is 0, then and is set to The same value. If S[j] is 1, and are assigned random values ​​while ensuring that their sum is equal to We then use keys M1 and M2 to further and Encrypt and obtain the ciphertext vector and Spliced ​​together, we get the final encrypted 770-dimensional document vector The client will be encrypted document vector Enc (h d ) to the server.

[0046] Step 4. The server stores the encrypted document vector in the open source vector database FAISS and constructs the corresponding ANN index I = Enc (h d ).

[0047] Step 5. The client uses the model to generate a 384-dimensional query vector representation h for the query to be searched q , and then use the generated key to encrypt the query vector. The encryption process is as follows: First, create a 385-dimensional vector Expressed as (rh q ,r). Then, based on the key S, the query vector Each dimension of is randomly split into two random vectors Specifically, if S[j] is 0, then and are assigned random values ​​while ensuring that their sum is equal to If S[j] is 1, and is set to We then use keys M1 and M2 to further and Encrypt and obtain the ciphertext vector and Spliced ​​together, we get the final encrypted 770-dimensional query trapdoor

[0048]

[0049] Step 6. The client uploads the query trapdoor. After receiving the trapdoor, the server uses the vector database FAISS to calculate the inner integral score between the trapdoor and the encrypted document index to perform a sorted search. According to the ascending order of the inner integral score, the server returns the query results that meet the requirements. Since the vector database can accelerate the search process under large-scale data, the vector database built in this solution significantly improves the retrieval efficiency of ciphertext data.

[0050] The client decrypts the result using the key.

[0051] Although the specific contents, implementation algorithms and drawings of the present invention are disclosed for the purpose of illustration, the purpose is to help understand the contents of the present invention and implement them accordingly, it can be understood by those skilled in the art that various substitutions, changes and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. The present invention should not be limited to the contents disclosed in the best embodiment of this specification and the drawings, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A context-enhanced semantic searchable encryption method, the steps of which include: 1) Use the labeled query-document pair data to train the selected model, obtain the optimal model and deploy it on the client; 2) The client generates a key {S, M1, M2}, where S is a randomly generated (m+1)-dimensional bit vector, M1 and M2 are reversible matrices of size (m+1)×(m+1), and m is a fixed vector dimension generated by the optimal model; 3) The client uses the optimal model to generate an m-dimensional document vector representation h for the document to be uploaded d , and then use the key {S,M1,M2} to represent the document vector h d Encrypt and obtain the encrypted document vector Enc(h d ) and upload it to the server; 4) The server stores the document vector Enc(h d ) and construct the corresponding index I = Enc(h d ); 5) The client generates an m-dimensional query vector representation h for the input query through the optimal model q , and then use the key {S,M1,M2} to represent the query vector h q Encrypt and obtain the final encrypted query trapdoor T q And upload to the server; 6) Server-side calculation query trap T q The similarity score between the result and index I is selected, and the top k results with the highest similarity scores are returned to the client.

2. The method according to claim 1, characterized in that The selected model is a pre-trained language model based on Transformer.

3. The method according to claim 1 or 2, characterized in that: Each training data contains a query-document pair and its corresponding label information; in the process of training the selected model using the training data, the distance between the query and the relevant documents is minimized by calculating the contrast loss, and the query vector representation and the document vector representation are optimized by contrastive learning, so that the distance between the query and the relevant documents is closer and the distance to the irrelevant documents is farther.

4. The method according to claim 1, characterized in that: The server will encrypt the document vector Enc(h d ) is stored in the open source vector database FAISS.

5. The method according to claim 1, characterized in that The server uses the vector database to calculate the query trap door T q and the similarity score between the encrypted document index I.

6. The method according to claim 1, characterized in that The client uses the key {S, M1, M2} to decrypt each returned result.

7. A context-enhanced semantically searchable encryption system, characterized in that: Including client and server; The client is used to generate a key {S, M1, M2}, train a Transformer-based pre-trained language model using labeled query-document pair data to obtain an optimal model; and generate an m-dimensional document vector representation h for the document to be uploaded using the optimal model. d , and then use the key {S,M1,M2} to represent the document vector h d Encrypt and obtain the encrypted document vector Enc(h d ) and upload it to the server; and generate an m-dimensional query vector representation h for the input query through the optimal model q , and then use the key {S,M1,M2} to represent the query vector h q Encrypt and obtain the final encrypted query trapdoor T q and upload it to the server; where S is a randomly generated (m+1)-dimensional bit vector, M1 and M2 are reversible matrices of size (m+1)×(m+1), and m is a fixed vector dimension generated by the optimal model; The server is used to store the document vector Enc(h d ) and construct the corresponding index I = Enc(h d ); and calculate the received query trapdoor T q The similarity score between the result and index I is selected, and the top k results with the highest similarity scores are returned to the client.

Images