A document search method, device, system, electronic device and storage medium
Patent Information
- Application Number
- CN202310525057.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-10
- Publication Date
- 2026-09-18
- Estimated Expiration
- 2043-05-10
AI Technical Summary
基于链表结构的多关键字动态排序可搜索加密方案在搜索时将搜索陷门与正排表中的文档逐一匹配并计算相关性得分,搜索时间复杂度与文档数量呈线性关系,查询效率非常低
[0042]As can be seen, when the present invention receives a search token sent by the client, it first extracts the target keyword searched by the user from the token, and uses an inverted index table to find the target document identifier corresponding to the target document containing the target keyword. The inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value. Therefore, using the inverted index table, all target documents containing the target keyword can be quickly identified. Furthermore, the present invention also uses a forward index table to determine the target first weight corresponding to the target keyword in the target document, and uses the target first weight to determine the relevance value of each target document. The forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document as the value. Therefore, based on the forward index table, the relevance between each target document and the target keyword searched by the user can be determined. Subsequently, the present invention can obtain target documents from the document set according to the target document identifier, sort each target document according to the relevance value, and then return the target documents to the client in an orderly manner, thereby improving the efficiency and flexibility of document retrieval. The present invention also provides a document search device, system, electronic device, and computer-readable storage medium, which have the above-mentioned beneficial effects.
Smart Images

Figure CN116644146B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of document search, and in particular to a document search method, apparatus, system, electronic device, and computer-readable storage medium. Background Technology
[0002] With the advent of the big data era and the widespread adoption of cloud storage services, users often choose to encrypt and store their data in the cloud to save on local data management costs. However, when users need to access this data, they face the problem of retrieval of encrypted data in cloud storage. Searchable encryption technology can effectively resolve the contradiction between data confidentiality and ease of use in the cloud environment, allowing cloud servers to perform search operations without decryption and return only a subset of the queried results. Clients only need to decrypt this subset to obtain the retrieved information. Among related technologies, the key to constructing a searchable encryption scheme for document-type data in cloud storage lies in the design of the index structure. Currently, the mainstream indexing technology supporting multi-keyword dynamic sorting is the forward index with a linked list structure. Searchable encryption schemes based on linked list structures match the search trapdoor with each document in the forward index and calculate relevance scores during the search. The search time complexity is linearly related to the number of documents, resulting in very low query efficiency.
[0003] Therefore, how to provide a document search method to improve document search efficiency is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention
[0004] The purpose of this invention is to provide a document search method, apparatus, system, electronic device, and computer-readable storage medium that can improve document search efficiency by utilizing inverted and forward indexes.
[0005] To address the aforementioned technical problems, this invention provides a document search method applied to a server, the method comprising:
[0006] When a search token sent by the client is received, the target keyword is extracted from the search token, and the inverted index table is used to find the target document identifier corresponding to the target document containing the target keyword; the inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value;
[0007] The target first weight corresponding to the target keyword in the target document is determined using the forward index table, and the relevance value of each target document is determined using the target first weight; the forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document in the specified document as the value.
[0008] The target document is obtained from the document collection according to the target document identifier, the target documents are sorted according to the relevance value of each target document, and the sorted target documents are returned to the client.
[0009] Optionally, the step of using an inverted index table to find the target document identifier corresponding to the target document containing the target keyword includes:
[0010] Determine whether the number of the target keywords is 1;
[0011] If so, then search for the target inverted index entry corresponding to the target keyword in the inverted index table, and obtain the target document identifier from the target inverted index entry;
[0012] If not, then search for candidate inverted index entries corresponding to each target keyword in the inverted index table, and obtain the target document identifier from the intersection of all candidate inverted index entries.
[0013] Optionally, the document identifier in the inverted index item is labeled with the first weight of the specified keyword in the document corresponding to the document identifier, and has been sorted according to the labeled first weight. After obtaining the target document identifier from the target inverted index item, the method further includes:
[0014] The target document is obtained from the document set according to the target document identifier, the target documents are sorted according to the order of each target document identifier in the target inverted index, and the sorted target documents are returned to the client.
[0015] Optionally, before receiving the search token sent by the client, the following steps are also included:
[0016] The receiving end sends a document update token; the document update token includes a document identifier to be updated, a document to be updated, a keyword sequence to be updated, and a document vector to be updated; the keyword sequence to be updated contains all the keywords in the document to be updated; the document vector to be updated is constructed based on a document vector template and contains the first weight of all the keywords in the document to be updated in the document to be updated; the initial value of each position in the document vector template is zero, and each position corresponds to each keyword in all documents;
[0017] Save the document identifier to be updated and the document to be updated to the document collection;
[0018] Create a new forward index entry in the forward index table, and save the document identifier to be updated, the keyword sequence to be updated, and the document vector to be updated into the new forward index entry;
[0019] The keywords in the keyword sequence to be updated are set as the keywords to be updated, and the document identifier to be updated is added to the inverted index entry corresponding to the keyword to be updated in the inverted index table.
[0020] Optionally, it also includes:
[0021] Receive the document identifier to be deleted sent by the data owner;
[0022] Based on the document identifier to be deleted, determine the corresponding forward index entry to be deleted for the document to be deleted in the forward index table;
[0023] Set the inverted index entry corresponding to the keyword in the forward index entry to be deleted in the inverted index table as the inverted index entry to be updated, and remove the document identifier to be deleted from the inverted index entry to be updated;
[0024] Delete the forward index entry to be deleted, and delete the document corresponding to the document identifier to be deleted in the document set.
[0025] Optionally, the search token includes a search vector constructed based on the document vector template and includes a second weight for the target keyword. The step of determining the target first weight corresponding to the target keyword in the target document using the forward index table, and determining the relevance value of each target document using the target first weight, includes:
[0026] The inner product between the search vector and the document vector contained in the forward index of each target document is calculated to obtain the relevance value of each target document.
[0027] Optionally, the first weight in the document vector is the TF value of the corresponding keyword in the corresponding document, and the second weight in the search vector is the IDF value of the corresponding target keyword in all documents. The step of calculating the inner product between the search vector and the document vectors contained in the forward index entries of each target document to obtain the relevance value of each target document includes:
[0028] The inner product between the search vector and the document vector contained in the forward index of each target document is calculated to calculate the TF-IDF value of each target keyword in each target document. The TF-IDF values of each target keyword in the same target document are then summed to obtain the relevance value of each target document.
[0029] Optionally, the documents in the document collection have been encrypted using a first method; the target keywords in the search token and the keywords in the inverted index table have been encrypted using a second method; and the first weight in the forward index table has been encrypted using a third method.
[0030] The present invention also provides a document search device for use on a server, the device comprising:
[0031] The inverted search module is used to extract target keywords from the search token sent by the client when it receives the search token, and use the inverted index table to find the target document identifier corresponding to the target document containing the target keyword; the inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the index content;
[0032] The forward search module is used to determine the target first weight corresponding to the target keyword in the target document using the forward index table, and to determine the relevance value of each target document using the target first weight; the forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document in the specified document as the index content;
[0033] The search results output module is used to obtain the target document from the document collection according to the target document identifier, sort the target documents according to the relevance value of each target document, and return the sorted target documents to the client.
[0034] This invention also provides a document search system, comprising: a server and a client, wherein,
[0035] The server, upon receiving a search token sent by the client, extracts target keywords from the search token and uses an inverted index table to find the target document identifier corresponding to the target document containing the target keywords. The inverted index table uses the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the index content. The server uses a forward index table to determine the target first weight corresponding to the target keyword in the target document and uses the target first weight to determine the relevance value of each target document. The forward index table uses the document identifier of the specified document as the key and the first weight of the keywords contained in the specified document in the specified document as the index content. The server retrieves the target document from the document set according to the target document identifier, sorts the target documents according to their relevance values, and returns the sorted target documents to the client.
[0036] The client is configured to send the search token to the server and receive the sorted target documents returned by the server.
[0037] The present invention also provides an electronic device, comprising:
[0038] Memory, used to store computer programs;
[0039] A processor for executing the computer program to implement the document search method described above.
[0040] The present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the document search method described above.
[0041] This invention provides a document search method applied to a server. The method includes: upon receiving a search token sent by a client, extracting target keywords from the search token and using an inverted index table to find the target document identifier corresponding to a target document containing the target keywords; the inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value; using a forward index table to determine the target first weight corresponding to the target keyword in the target document, and using the target first weight to determine the relevance value of each target document; the forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document in the specified document as the value; retrieving the target document from a document set according to the target document identifier, sorting each target document according to the relevance value of each target document, and returning the sorted target documents to the client.
[0042] As can be seen, when the present invention receives a search token sent by the client, it first extracts the target keyword searched by the user from the token, and uses an inverted index table to find the target document identifier corresponding to the target document containing the target keyword. The inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value. Therefore, using the inverted index table, all target documents containing the target keyword can be quickly identified. Furthermore, the present invention also uses a forward index table to determine the target first weight corresponding to the target keyword in the target document, and uses the target first weight to determine the relevance value of each target document. The forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document as the value. Therefore, based on the forward index table, the relevance between each target document and the target keyword searched by the user can be determined. Subsequently, the present invention can obtain target documents from the document set according to the target document identifier, sort each target document according to the relevance value, and then return the target documents to the client in an orderly manner, thereby improving the efficiency and flexibility of document retrieval. The present invention also provides a document search device, system, electronic device, and computer-readable storage medium, which have the above-mentioned beneficial effects. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0044] Figure 1 This is a schematic diagram of a document search system provided in an embodiment of the present invention;
[0045] Figure 2 A flowchart illustrating a document search method provided in an embodiment of the present invention;
[0046] Figure 3 This is a schematic diagram of an inverted index table provided in an embodiment of the present invention;
[0047] Figure 4 This is a schematic diagram of a forward index table provided in an embodiment of the present invention;
[0048] Figure 5 This is a schematic diagram of another inverted index table provided in an embodiment of the present invention;
[0049] Figure 6 A timing diagram of a document search method provided in an embodiment of the present invention;
[0050] Figure 7 This is a structural block diagram of a document search device provided in an embodiment of the present invention;
[0051] Figure 8 This is a structural block diagram of a document search system provided in an embodiment of the present invention. Detailed Implementation
[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0053] In related technologies, the key to constructing a searchable and encrypted scheme for document-type data in cloud storage lies in the design of the index structure. Currently, the mainstream indexing technologies supporting dynamic sorting with multiple keywords are linked list-based forward indexes and tree-based forward indexes. Searchable and encrypted schemes based on linked list-based dynamic sorting match the search trapdoor with each document in the forward index and calculates relevance scores. The search time complexity is linearly related to the number of documents, resulting in very low query efficiency. Therefore, this invention provides a document search method that can improve document search efficiency by utilizing inverted and forward indexes.
[0054] To facilitate understanding, the system used in this invention will first be described. Please refer to... Figure 1 , Figure 1This diagram illustrates a document search system provided in an embodiment of the present invention. The system comprises three main entities: a data owner (hereinafter referred to as the data owner end), a data user (hereinafter referred to as the client), and a semi-trustworthy cloud server (hereinafter referred to as the server end). To facilitate data storage and retrieval, the data owner end can upload encrypted documents, encrypted index information, and a dictionary tag set (containing all keywords of the document, already encrypted) to the server end. It then sends the system key used for encryption, the dictionary (containing all keywords of the document), and relevant parameters required for the query (e.g., inverse document frequency) to the client via a secure tunnel. The client can then construct a search request based on the dictionary and relevant parameters and send it to the server end. The server end can perform relevant searches in the secure index based on the search request, obtain the encrypted document based on the search results, and return the encrypted document to the client in an orderly manner. Finally, the client can decrypt the encrypted document using the existing system key to obtain the plaintext document and complete the search. It is evident that the most efficiency-critical step in the search process is the server-side retrieval of the encrypted document based on the secure index. If the secure index cannot guarantee high search efficiency, it can easily negatively impact the user experience. Therefore, this invention mainly improves the security index by simultaneously introducing inverted and forward indexes to enhance retrieval efficiency. It should be noted that the embodiments of this invention do not limit other implementation details of the system described above, such as the number of clients, data ownership terminals, and the specific terminal forms of the clients and servers; these can be set according to actual application requirements.
[0055] Based on the above system description, the document search method provided by the embodiments of the present invention will be described in detail below. Please refer to... Figure 2 , Figure 2 A flowchart of a document search method provided in an embodiment of the present invention, the method being applied to a server, may include:
[0056] S201. When a search token sent by the client is received, the target keyword is extracted from the search token, and the target document identifier corresponding to the target document containing the target keyword is found using the inverted index table. The inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value.
[0057] In this embodiment of the invention, the search request sent by the client is specifically a search token, which contains the target keywords queried by the user. It should be noted that the data owner, client, and server all possess all the keywords of the document. The difference lies in that the data owner and client can possess plaintext keywords, while the server only possesses encrypted keywords. This avoids the server accessing sensitive information in the document. Specifically, the data owner and client can store all the keywords of the document in the form of a dictionary, while the server can store all the encrypted keywords of the document in the form of tags. Furthermore, the client can use its dictionary to provide query services to the user and can also use the dictionary to construct the aforementioned search token. It is understood that, to avoid the server accessing sensitive information, the target keywords in the search token can also be encrypted, so that the server can only complete all retrieval tasks under encrypted conditions. Of course, it should be noted that, to meet other query needs, the search token may contain other information besides the target keywords, such as search vectors related to the target keywords, to improve the server's retrieval efficiency. For details, please refer to the description in the following embodiments.
[0058] Furthermore, after obtaining the search token, the server can extract the target keyword from the token and use an inverted index table to find the target document identifier corresponding to the target document containing the target keyword. The inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value. The specified keyword specifically includes all keywords contained in all documents; that is, each keyword corresponds to one inverted index entry in the inverted index table. Please refer to [reference needed] for a possible form of the inverted index table. Figure 3 , Figure 3 This is a schematic diagram of an inverted index table provided in an embodiment of the present invention. Wherein, w1', w2', w... m 'All represent encrypted keywords, FID 11 FID 12 , This represents the document identifier corresponding to the document containing the keyword w1', i.e., w1', FID. 11 FID 12 , This forms an inverted index entry; FID 21 FID 22 , The document identifier (FID) represents the document containing the keyword w2'. m1 FID m2 , Indicates that w mThe document identifier corresponding to the corresponding keyword is listed below, and so on. It is evident that, based on the inverted index table, the server can quickly retrieve the target document identifiers corresponding to each target document containing the target keyword without needing to perform a complete traversal search in the forward index table, significantly improving retrieval efficiency. It should be noted that, in this embodiment of the invention, the inverted index entries specifically use a linked list to store each document identifier, which improves the efficiency of updating and deleting document identifiers; furthermore, the linked list structure facilitates the sorting of document identifiers, thereby meeting other retrieval requirements. Of course, the inverted index entries can also use other forms to store document identifiers, which can be set according to actual application needs.
[0059] It should be noted that the retrieval methods for single-target keywords and multi-target keywords differ in the inverted index table. Specifically, for a single-target keyword, all target document identifiers can be directly retrieved from its corresponding inverted index entry; however, for multi-target keywords, it is necessary to first set the inverted index entry corresponding to each target keyword as a candidate inverted index entry, then take the intersection of these candidate inverted index entries, and finally obtain the target document identifier from this intersection. It can be understood that the target document identifiers appearing in the intersection have already appeared in all the aforementioned candidate inverted index entries; in other words, the target documents corresponding to the target document identifiers in the intersection simultaneously contain these multiple target keywords.
[0060] Based on this, using an inverted index table to find the target document identifier corresponding to a target document containing the target keyword can include:
[0061] Step 11: Determine if the number of target keywords is 1; if yes, proceed to step 12; if no, proceed to step 13.
[0062] Step 12: Locate the target inverted index entry corresponding to the target keyword in the inverted index table, and obtain the target document identifier from the target inverted index entry;
[0063] Step 13: Find the candidate inverted index entries corresponding to each target keyword in the inverted index table, and obtain the target document identifier from the intersection of all candidate inverted index entries.
[0064] S202. Use the forward index table to determine the target first weight of the target keyword in the target document, and use the target first weight to determine the relevance value of each target document; the forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document in the specified document as the value.
[0065] After completing the relevant retrieval of the inverted index table, this embodiment of the invention also needs to use the forward index table for the next round of retrieval. Specifically, the forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keywords contained in the specified document as the value, where the first weight represents the importance of the keyword in the specified document. In this way, this embodiment of the invention can determine the weight of the target keyword in each target document, and then only needs to sum the weights of each target keyword in the same document to determine the relevance value of the document, thus determining the association between the document and all target keywords. It should be noted that this embodiment of the invention does not limit the specific form of the first weight. For example, it can be the term frequency of the keyword in the document, or it can be the TF-IDF value (Term Frequency-Inverse Document Frequency) of the keyword in the document, which can be set according to the actual application requirements. It should also be noted that the specified document can specifically include all documents, that is, each document can correspond to a forward index entry in the forward index table.
[0066] S203. Obtain the target document from the document collection based on the target document identifier, sort the target documents according to their relevance values, and return the sorted target documents to the client.
[0067] After a relevant search of the complete inverted index table, this embodiment of the invention can retrieve target documents from the document collection according to the target document identifier, sort the target documents according to their relevance, and finally return the sorted target documents to the client. The document collection is used to store encrypted documents and their corresponding document identifiers. It is understood that the sorting method for each target document can be descending, i.e., the most relevant documents are placed first, and the least relevant documents are placed last. Of course, after sorting, a preset number of target documents can also be returned to the client to filter out some irrelevant target documents. It should be noted that this embodiment of the invention does not limit the specific value of the preset number and can be set according to actual application needs.
[0068] Furthermore, for single-target keyword retrieval, although using two index tables has improved the retrieval effect compared to related technologies, there is still room for further optimization. In this regard, embodiments of the present invention can further optimize the inverted index entries. Specifically, each document identifier in the inverted index entry can be labeled with the first weight of the specified keyword in the document corresponding to that document identifier. The identifiers can then be sorted based on their first weights to characterize the importance of the specified keyword in each document. The sorting method can be: document identifiers corresponding to documents with higher first weights are ranked first, while those corresponding to documents with lower first weights are ranked last. Therefore, after completing the retrieval using the inverted index table, not only can each target document identifier be obtained using the inverted index entry corresponding to the target keyword, but the relevance ranking of each target document identifier can also be obtained. This can then be directly used for ranking target documents, thereby improving the retrieval efficiency of target documents in the case of a single target keyword. Based on this, the document identifiers in the inverted index entry are labeled with the first weight of the specified keyword in the document corresponding to the document identifier and have been sorted according to the labeled first weights. After obtaining the target document identifiers from the target inverted index entry, the following steps are also included:
[0069] S204. Obtain the target document from the document collection based on the target document identifier, sort the target documents according to the order of each target document identifier in the target inverted index, and then return the sorted target documents to the client.
[0070] It should be noted that, to prevent the data server from accessing sensitive information, the first weight in the inverted index item can also be encrypted. It is important to note that this embodiment of the invention must ensure that the encrypted first weight in the inverted index item can still be used for sorting. Therefore, this embodiment of the invention can utilize Order-Preserving Encoding (OPE) to encrypt the first weight in the inverted index item, thereby preserving the sortability of the first weight. For another possible form of the inverted index table, please refer to... Figure 5 , Figure 5 This is a schematic diagram of another inverted index table provided in an embodiment of the present invention. Wherein, w1', w2', w... m 'All represent encrypted keywords, FID 11 FID 12 , This represents the document identifier (TF) corresponding to the document containing the keyword w1'. 11 '、TF 12 '、 Indicates w1' in FID 11 FID 12 , The encrypted value corresponds to the first weight (e.g., word frequency) in the document, and so on.
[0071] Based on the above embodiments, when the present invention receives a search token sent by the client, it first extracts the target keywords searched by the user from the token, and uses an inverted index table to find the target document identifier corresponding to the target document containing the target keywords. The inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value. Therefore, using the inverted index table, all target documents containing the target keywords can be quickly identified. Furthermore, the present invention also uses a forward index table to determine the target first weight corresponding to the target keyword in the target document, and uses the target first weight to determine the relevance value of each target document. The forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document as the value. Therefore, based on the forward index table, the relevance between each target document and the target keywords retrieved by the user can be determined. Subsequently, the present invention can obtain target documents from the document set according to the target document identifier, sort each target document according to the relevance value, and then return the target documents to the client in an orderly manner, thereby improving the efficiency and flexibility of document retrieval.
[0072] Based on the above embodiments, the creation and updating of inverted index tables, forward index tables, and document collections will be described in detail below. In one possible case, before receiving the search token sent by the client, this method may further include:
[0073] S301, Receive a document update token sent by the data owner; the document update token includes a document identifier to be updated, a document to be updated, a keyword sequence to be updated, and a document vector to be updated; the keyword sequence to be updated contains all the keywords in the document to be updated; the document vector to be updated is constructed based on a document vector template and contains the first weight of all keywords in the document to be updated; the initial value of each position in the document vector template is zero, and each position corresponds to each keyword in all documents.
[0074] The data owner can update the inverted index, forward index, and document set by sending a document update token to the server. This document update token specifically includes a document identifier to be updated, the document to be updated, a sequence of keywords to be updated, and a document vector to be updated. The sequence of keywords to be updated contains all keywords corresponding to the document to be updated, while the document vector to be updated contains the first weight corresponding to each keyword in the document to be updated. It should be noted that the document vector and the search vector mentioned above need to be used together to improve the efficiency of generating relevance values. Specifically, to ensure that the document vector and the search vector can be directly multiplied, these two vectors can be generated based on the same document vector template, where each position in the document vector template corresponds to each keyword in all documents, and the initial value of each position is zero. Furthermore, when generating a document vector, the position corresponding to a keyword in the document in the document vector template can be found, and the first weight corresponding to that keyword in the document can be written into that position; when generating a search vector, the position corresponding to the target keyword in the document vector template can be found, and the relevant parameters of the target keyword can be written into that position. Furthermore, when performing the inner product operation between the document vector and the search vector, since the positions in the search vector corresponding to the target keyword contain data, while the positions not corresponding to the target keyword do not, performing the inner product operation between the search vector and the document vector not only directly filters out all data related to the target keyword, but also allows multiplying and summing these data to obtain the relevance value of the target document. In other words, by setting the above vectors, the retrieval of data related to the target keyword can be avoided, thereby improving the efficiency of relevance value calculation.
[0075] The following two examples illustrate the detailed process of calculating document relevance values using document vectors and search vectors. In one scenario, the data recorded at the corresponding position of a keyword in the document vector is the first weight; the data recorded at the corresponding position of the target keyword in the search vector is 1. Taking the inner product of the document vector and the search vector yields the sum of the first weights of all target keywords, which can then be used as the document's relevance value. In another scenario, the data recorded at the corresponding position of a keyword in the document vector is the first weight; the data recorded at the corresponding position of the target keyword in the search vector is the second weight. Taking the inner product of the document vector and the search vector yields the sum of the products of the first and second weights of all target keywords, which can then be used as the document's relevance value.
[0076] S302. Save the document identifier to be updated and the document to be updated to the document collection.
[0077] Understandably, during the initialization phase, when the data owner sends the document update token for the first time, the server can first create a document collection and then write data into that collection.
[0078] S303. Create a new forward index entry in the forward index table, and save the document identifier to be updated, the keyword sequence to be updated, and the document vector to be updated to the new forward index entry.
[0079] As can be seen, each forward index entry in the forward index table stores a document identifier, a keyword sequence, and a document vector. A possible form of the forward index table can be found in [reference needed]. Figure 4 , Figure 4 This is a schematic diagram of a forward index table provided in an embodiment of the present invention. It can be understood that during the initialization phase, i.e., when the data owner first sends a document update token, the server can first create a forward index table and then write data into that index table.
[0080] S304. Set the keywords in the keyword sequence to be updated as the keywords to be updated, and add the document identifier to be updated to the inverted index entry corresponding to the keyword to be updated in the inverted index table.
[0081] Understandably, during the initialization phase, when the data owner sends the document update token for the first time, the server can first create an inverted index table and then write data into that index table. Similarly, it is understandable that if the keywords in the keyword sequence to be updated do not have corresponding inverted index entries in the inverted index table, then an inverted index entry can be created first, and then the keywords and the document identifier to be updated can be written into that inverted index entry.
[0082] As can be seen, since both a forward index table and an inverted index table are set up simultaneously, the embodiments of the present invention can quickly update the inverted index table based on the forward index table, thereby improving the dynamic update efficiency of the two tables.
[0083] The following section details the methods for deleting related data in inverted indexes, forward indexes, and document collections. In one possible scenario, this method may also include:
[0084] S401, Receive the document identifier to be deleted sent by the data owner;
[0085] S402. Determine the corresponding forward index entry for the document to be deleted in the forward index table based on the document identifier to be deleted;
[0086] S403. Set the inverted index item corresponding to the keyword in the forward index item to be deleted in the inverted index table as the inverted index item to be updated, and remove the document identifier to be deleted from the inverted index item to be updated.
[0087] S404. Delete the forward index item to be deleted and delete the document corresponding to the document identifier to be deleted in the document collection.
[0088] As can be seen, since both a forward index table and an inverted index table are set up simultaneously, the embodiments of the present invention can quickly delete relevant data in the inverted index table based on the forward index table, thereby improving the dynamic data deletion efficiency of the two tables.
[0089] Based on the above embodiments, the specific calculation method for the relevance value of the target document is described in detail below. In one possible case, the search token contains a search vector, which is constructed based on a document vector template and includes a second weight for the target keyword. The first weight corresponding to the target keyword in the target document is determined using a forward index table, and the relevance value of each target document is determined using the first target weight. This can include:
[0090] S501. Calculate the inner product between the search vector and the document vector contained in the forward index of each target document to obtain the relevance value of each target document.
[0091] As mentioned above, the search vector and document vector are constructed using the same document vector template. Therefore, these two vectors can be directly multiplied and summed to obtain the relevance value of the target document. The search vector specifically contains the second weight for the target keyword. In one possible scenario, the first weight can be a TF value, and the second weight can be an IDF value. Thus, when the search vector is multiplied by the document vector, the sum of the TF-IDF values of all target keywords can be obtained. This sum of TF-IDF values can then be used to describe the relevance between the target document and the target keyword.
[0092] Based on this, the first weight in the document vector is the TF value of the corresponding keyword in the corresponding document, and the second weight in the search vector is the IDF value of the corresponding target keyword in all documents. The inner product between the search vector and the document vectors contained in the forward index entries of each target document is calculated to obtain the relevance value of each target file, including:
[0093] Step 21: Calculate the inner product between the search vector and the document vector contained in the forward index of each target document to calculate the TF-IDF value of each target keyword in each target document, and sum the TF-IDF values of each target keyword in the same target document to obtain the relevance value of each target document.
[0094] It should be noted that when the first weight is the TF value, the data owner should be responsible for calculating the TF value. Specifically, before generating a document update token, the data owner should calculate the TF value corresponding to the keywords in the document to be updated within that document, and use these TF values to generate the document vector to be updated. Furthermore, when the second weight is the IDF value, the calculation of the IDF value can be performed by either the data owner or the server. For example, when processing a document to be updated, the data owner updates the IDF value of the keywords in the document to be updated across all documents; or, for example, after updating the keywords and corresponding document identifiers to the inverted index table, the server can calculate the IDF value using the number of document identifiers currently corresponding to the keywords and the total number of document identifiers. Furthermore, regardless of whether the IDF value is calculated by the data owner or the server, it needs to be periodically pushed to the client so that the client can generate search vectors.
[0095] Based on the above embodiments, to prevent the server from accessing sensitive data, the relevant data in the document collection, forward index table, and inverted index table have been encrypted. Specifically, the documents in the document collection have been encrypted using a first method; the target keywords in the search token and the keywords in the inverted index table have been encrypted using a second method; and the first weight in the forward index table has been encrypted using a third method. Furthermore, if the entries in the inverted index table also contain the first weight, the first weight in the inverted index table can be encrypted using a fourth method. Preferably, the first method can be a symmetric encryption scheme SKE (Secret Key Encryption) resistant to indistinguishable chosen plaintext attacks, the second method can be a pseudo-random function, the third method can be Secure KNN (Secure Nearest Neighbor Encryption), and the fourth method can be Order-Preserving Encoding (OPE). It should be particularly noted that Secure KNN is specifically used to encrypt the document vectors in the forward index table, while order-preserving encryption is specifically used to encrypt a single first weight.
[0096] The document search method described above is explained below based on a specific time sequence diagram. Please refer to [the provided text]. Figure 6 , Figure 6 The following is a timing diagram of a document search method provided in an embodiment of the present invention. The searchable encryption method includes the following steps:
[0097] 1. System initialization and key generation (K, γ) F ,γ w ,C)←gen(1 λThe system selects the following symmetric encryption schemes to resist indistinguishable chosen-plaintext attacks: SKE = (gen, enc, dec), SecurekNN encryption scheme KNN = (gen, enc, dec), order-preserving encryption scheme OPE = (gen, enc, dec), and pseudo-random function f, where gen represents the generator function, enc represents the encryption function, and dec represents the decoder function.
[0098] 1.1 The data owner generates a system key K = (k1, k2, k3, k4) based on the security parameter λ. Where k1 is used with f; k2 is used with SKE; k3 is used with KNN; and k4 is used with OPE.
[0099] 1.2. The data owner calculates the label set W' = {w1', w2', ..., w...} of the ordered dictionary set W. m '} and upload it to the server, where
[0100] 1.3 The data owner sends the ordered dictionary set W, inverse document frequency information, and system key K to the client through a secure channel;
[0101] 1.4 The server generates an empty set R and two empty index tables γ. F γ w , where γ w With ordered dictionary tags.
[0102] 2. The data owner generates and adds a document token. For document sets Each document F i =(P i ,KW i ), The data owner assigns a document identifier (FID). i And do the following:
[0103] 2.1 Constructing Document Vectors: Using a vector space model, an m-dimensional normalized document vector FV is constructed for each document based on a dictionary W. i Document F i Keyword set KW i Includes w j At that time, the component at the corresponding position Otherwise, the component is set to 0.
[0104] 2.2 Encrypted Document Vectors: KNN is used to encrypt document vectors, i.e., FV. i =KNNenc(FV i ,k3);
[0105] 2.3 Encrypted word frequency: The word frequency is encrypted using an order-preserving encryption method, that is:
[0106] TF ij =OPEenc(TF) ij ,k4);
[0107] 2.4 Encrypted Documents: Documents are encrypted using SKE, i.e., P... i '=SKEenc(P i ,k2);
[0108] 2.5. Generate encrypted document information:
[0109]
[0110] The data owner will add a document token T. a ={CF1,CF2,…,CF n Upload to the server.
[0111] 3. Add a document (γ) to the server. F ',γ w ',C')←add(T a ,γ F ,γ w C): After receiving the document addition token from the data owner, the server performs the following operations:
[0112] 3.1 Update the forward index table γ F In the forward index table γ F Add n rows, that is:
[0113] The updated forward index table is γ F ';
[0114] 3.2 Update the inverted index table γ w For each encrypted document information CF i Each keyword tag in w ij ', The server searches the inverted index table γ w The dictionary label, if w ij 'is γ w The key value will then be FID i ||TF ij Add to γ w [w ij ′], and in accordance with TF ij The size of ' remains in descending order; if w ij 'Not γ' w If the key value is w, then w ij 'Add to γ w In the key-value pair, and put FID i ||TF ij Add to γw [w ij In '] and according to TF ij The size of ' remains in descending order, and the updated inverted index table is γ. w ';
[0115] 3.3 Update the ciphertext document set C: ciphertext P of n documents i 'Add to C to get C'.
[0116] 4. Server-side document deletion (γ) F ',γ w ',C')←delete(FID,γ F ,γ w C):
[0117] 4.1 The data owner sends the document identifier FID corresponding to the document F to be deleted to the server;
[0118] 4.2 The server searches for the forward index γ based on the FID. F To obtain the document (let's assume it's document F) i The corresponding keyword tag set Query the inverted index γ based on the elements of this set. w And delete the FID in the corresponding linked list. i The updated search index table is γ w ';
[0119] 4.3 The server starts from the ascending index table γ F Remove FID i The corresponding document information, the updated document index table is γ F '; Remove P from the encrypted document set C i The updated encrypted document set is C'.
[0120] 5. The client generates a search token T. Q ←searchToken(K,Q): The client's query statement Q, after preprocessing to extract keywords, yields the query keyword set W. Q ={w1, w2, ..., w s}, s≥1, then perform the following operations:
[0121] 5.1 Constructing the query vector: A vector space model is adopted, based on the dictionary W and the corresponding IDF. w Construct an m-dimensional normalized query vector QV, where the keyword w corresponds to the position. j In the query keyword set W Q When in the middle, its weight All other components are set to 0; when s = 1, there is no need to construct a query vector, proceed directly to 5.3;
[0122] 5.2 Encrypted Query Vector: The query vector is encrypted using KNN, i.e., QV' = KNNenc(QV, k3);
[0123] 5.3 Calculate keyword tags: Use a pseudo-random function f to calculate the query keyword tags, i.e.:
[0124]
[0125] The client will search for token T Q ={w1',w2',…,w s Upload ',QV'} to the server.
[0126] 6. Server-side search for text I Q ←search(T Q ,γ F ,γ w ): The server creates an empty set I Q And complete the following operations:
[0127] 6.1 According to T Q Search the inverted index one by one using the keyword tags in the index. w Find the intersection of the document identifier sets corresponding to the s keyword tags;
[0128] 6.2. Query the forward index table γ based on the document identifier in the intersection. F Calculate the inner product of the query vector and the corresponding document vector to obtain the document relevance score, i.e.:
[0129]
[0130] in It is the normalized word frequency. It is the normalized inverse document frequency, W F This is the keyword set for document F. Skip 6.2 when s = 1.
[0131] 6.3 Sort the documents in the intersection set according to the scores, and extract the encrypted text of the corresponding documents in order and store it in I. Q The result is then returned to the querying user's client.
[0132] 7. Client-side decryption of document P Q ←dec(K,I Q ): Client to I Q The documents in the file are decrypted sequentially, i.e., P i =SKEdec(P i ′,k2), to obtain the final plaintext result set.
[0133] The following describes the document search device, system, electronic device, and computer-readable storage medium provided in the embodiments of the present invention. The document search device, system, electronic device, and computer-readable storage medium described below can be referred to in correspondence with the document search method described above.
[0134] Please refer to Figure 7 , Figure 7 This is a structural block diagram of a document search device provided in an embodiment of the present invention. The device is applied to a server and may include:
[0135] The inverted search module 701 is used to extract target keywords from the search token sent by the client when it receives the search token, and use the inverted index table to find the target document identifier corresponding to the target document containing the target keyword; the inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the index content.
[0136] The forward search module 702 is used to determine the target first weight of the target keyword in the target document using the forward index table, and to determine the relevance value of each target document using the target first weight; the forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document in the specified document as the index content;
[0137] The search results output module 703 is used to retrieve target documents from the document collection based on the target document identifier, sort the target documents according to their relevance values, and return the sorted target documents to the client.
[0138] Optionally, the inverted search module 701 may include:
[0139] The decision submodule is used to determine whether the number of target keywords is 1. If yes, it searches for the target inverted index entry corresponding to the target keyword in the inverted index table and obtains the target document identifier from the target inverted index entry. If no, it searches for the candidate inverted index entries corresponding to each target keyword in the inverted index table and obtains the target document identifier from the intersection of all candidate inverted index entries.
[0140] Optionally, the document identifiers in the inverted index entries are labeled with the first weight of a specified keyword in the document corresponding to the document identifier, and have been sorted according to the labeled first weight. The device may further include:
[0141] The sorting module is used to retrieve target documents from the document collection based on the target document identifier, sort the target documents according to the order of each target document identifier in the target inverted index, and then return the sorted target documents to the client.
[0142] Optionally, the device may further include:
[0143] The document update token receiving module is used to receive document update tokens sent by the data owner. The document update token includes an identifier of the document to be updated, the document to be updated, a sequence of keywords to be updated, and a vector of the document to be updated. The sequence of keywords to be updated contains all the keywords in the document to be updated. The vector of the document to be updated is constructed based on a document vector template and contains the first weight of all keywords in the document to be updated. The initial value of each position in the document vector template is zero, and each position corresponds to each keyword in all documents.
[0144] The document saving module is used to save the document identifier to be updated and the document to be updated to the document collection;
[0145] The forward index entry update module is used to create new forward index entries in the forward index table and save the document identifier to be updated, the keyword sequence to be updated, and the document vector to be updated to the new forward index entry;
[0146] The inverted index item update module is used to set the keywords in the keyword sequence to be updated as the keywords to be updated, and add the document identifier to be updated to the corresponding inverted index item of the keyword to be updated in the inverted index table.
[0147] Optionally, the device may further include:
[0148] The deletion information receiving module is used to receive the document identifier to be deleted sent by the data owner;
[0149] The document deletion module is used to determine the corresponding forward index entry to be deleted for the document to be deleted in the forward index table based on the document identifier to be deleted.
[0150] The forward index item deletion module is used to set the inverted index item corresponding to the keyword in the forward index item to be deleted as the inverted index item to be updated in the inverted index table, and remove the document identifier to be deleted from the inverted index item to be updated.
[0151] The inverted index item deletion module is used to delete the forward index item to be deleted and to delete the document corresponding to the document identifier to be deleted from the document collection.
[0152] Optionally, the search token contains a search vector constructed based on a document vector template and includes a second weight for the target keyword. The positive search module 702 may include:
[0153] The relevance calculation submodule is used to calculate the inner product between the search vector and the document vector contained in the forward index of each target document, so as to obtain the relevance value of each target file.
[0154] Optionally, the first weight in the document vector is the TF value of the corresponding keyword in the corresponding document, and the second weight in the search vector is the IDF value of the corresponding target keyword in all documents. The relevance calculation submodule may include:
[0155] The relevance calculation unit is used to calculate the inner product between the search vector and the document vector contained in the forward index of each target document, so as to calculate the TF-IDF value of each target keyword in each target document, and sum the TF-IDF values of each target keyword in the same target document to obtain the relevance value of each target document.
[0156] Optionally, the documents in the document collection have been encrypted using the first method; the target keywords in the search token and the keywords in the inverted index table have been encrypted using the second method; and the weights in the forward index table have been encrypted using the third method.
[0157] Please refer to Figure 8 , Figure 8 This is a structural block diagram of a document search system provided in an embodiment of the present invention. The system may include: a server 801 and a client 802, wherein...
[0158] Server-side 801, upon receiving a search token from the client, extracts the target keyword from the search token and uses an inverted index table to find the target document identifier corresponding to the target document containing the target keyword. The inverted index entries use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the index content. The forward index table is used to determine the target first weight corresponding to the target keyword in the target document, and the target first weight is used to determine the relevance value of each target document. The forward index entries use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document as the index content. The target document is retrieved from the document set based on the target document identifier, sorted according to the relevance value of each target document, and the sorted target documents are returned to the client.
[0159] Client 802 is used to send a search token to the server and receive the sorted target documents returned by the server.
[0160] Optionally, server-side 801 is also used to determine whether the number of target keywords is 1; if yes, then the target inverted index item corresponding to the target keyword is searched in the inverted index table, and the target document identifier is obtained from the target inverted index item; if no, then the candidate inverted index item corresponding to each target keyword is searched in the inverted index table, and the target document identifier is obtained from the intersection of all candidate inverted index items.
[0161] Optionally, the document identifiers in the inverted index table have been sorted according to the first weight of the specified keyword in each document containing the specified keyword. The server 801 is also used to obtain the target document from the document set according to the target document identifier, sort each target document according to the order of each target document identifier in the target inverted index item, and proceed to the step of returning the sorted target document to the client.
[0162] Optionally, the system also includes: a data ownership terminal 803, wherein,
[0163] Server 801 is also used to receive document update tokens sent by the data owner; the document update token includes a document identifier to be updated, a document to be updated, a keyword sequence to be updated, and a document vector to be updated; the keyword sequence to be updated contains all the keywords in the document to be updated; the document vector to be updated is constructed based on a document vector template and contains the first weight of all keywords in the document to be updated; the initial value of each position in the document vector template is zero, and each position corresponds to each keyword in all documents; the document identifier to be updated and the document to be updated are saved to the document collection; a new forward index entry is created in the forward index table, and the document identifier to be updated, the keyword sequence to be updated, and the document vector to be updated are saved to the new forward index entry; the keywords in the keyword sequence to be updated are set as the keywords to be updated, and the document identifier to be updated is added to the inverted index entry corresponding to the keyword to be updated in the inverted index table;
[0164] The data owner 803 is also used to construct document update tokens and send them to the server.
[0165] Optionally, server 801 is also used to receive a document identifier to be deleted sent by the data owner; determine the corresponding forward index entry to be deleted for the document to be deleted in the forward index table based on the document identifier to be deleted; set the inverted index entry corresponding to the keyword in the forward index entry to be deleted in the inverted index table as the inverted index entry to be updated, and remove the document identifier to be deleted from the inverted index entry to be updated; delete the forward index entry to be deleted, and delete the document corresponding to the document identifier to be deleted in the document collection;
[0166] The data owner also uses it to send the identifier of the document to be deleted to the server.
[0167] Optionally, the search token contains a search vector, which is constructed based on a document vector template and includes a second weight for the target keyword. The server 801 is also used to calculate the inner product between the search vector and the document vectors contained in the forward index of each target document to obtain the relevance value of each target file.
[0168] Optionally, the first weight in the document vector is the TF value of the corresponding keyword in the corresponding document, and the second weight in the search vector is the IDF value of the corresponding target keyword in all documents. The server 801 is also used to calculate the inner product between the search vector and the document vectors contained in the forward index of each target document, so as to calculate the TF-IDF value of each target keyword in each target document, and sum the TF-IDF values of each target keyword in the same target document to obtain the relevance value of each target document.
[0169] This invention also provides an electronic device that may include:
[0170] Memory, used to store computer programs;
[0171] A processor is used to implement the document search method described above when executing computer programs.
[0172] Since the embodiments of the electronic device part correspond to the embodiments of the document search method part, please refer to the description of the embodiments of the document search method part for the embodiments of the electronic device part, and they will not be repeated here.
[0173] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the document search method described in any of the above embodiments.
[0174] Since the embodiments of the computer-readable storage medium portion correspond to the embodiments of the document search method portion, the embodiments of the storage medium portion are described in the document search method portion and will not be repeated here.
[0175] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0176] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0177] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0178] The foregoing has provided a detailed description of the document search method, apparatus, system, electronic device, and storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.
Claims
1. A document search method, characterized in that, Applied to the server side, the method includes: When a search token sent by the client is received, the target keyword is extracted from the search token, and the target document identifier corresponding to the target document containing the target keyword is found using an inverted index table. The inverted index entries in the inverted index table use the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the value. The search token contains a search vector, which is constructed based on a document vector template and contains a second weight of the target keyword. The initial value of each position in the document vector template is zero, and each position corresponds to each keyword in all documents. When generating the search vector, the position corresponding to the target keyword in the document vector template is found, and the second weight of the target keyword is written into the position. The first weight of the target keyword in the target document is determined using a forward index table, and the relevance value of each target document is determined using the first weight. The forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document as the value. Each forward index entry stores the document identifier and a document vector. The document vector is constructed based on a document vector template and contains the first weight of all keywords in the document to be updated. When generating the document vector, the position of the keyword in the document is found in the document vector template, and the first weight of the keyword in the document is written into that position. The target document is obtained from the document set according to the target document identifier, and each target document is sorted according to its relevance value. The sorted target documents are then returned to the client. The first weight in the document vector is the TF value of the corresponding keyword in the corresponding document. The data owner is responsible for calculating the TF value. The document vector is updated by the data owner to the server. The second weight in the search vector is the IDF value of the corresponding target keyword in all documents. The IDF value is updated by the data owner and periodically pushed to the client. The process of determining the target first weight of the target keyword in the target document using the forward index table, and determining the relevance value of each target document using the target first weight, includes: Calculate the inner product between the search vector and the document vector contained in the forward index of each target document to calculate the TF-IDF value of each target keyword in each target document, and sum the TF-IDF values of each target keyword in the same target document to obtain the relevance value of each target document. The data owner and the client have plaintext keywords, while the server only has encrypted keywords. The relevant data in the document collection, the forward index table, and the inverted index table are encrypted.
2. The document search method according to claim 1, characterized in that, The step of using an inverted index table to find the target document identifier corresponding to the target document containing the target keyword includes: Determine whether the number of the target keywords is 1; If so, then search for the target inverted index entry corresponding to the target keyword in the inverted index table, and obtain the target document identifier from the target inverted index entry; If not, then search for candidate inverted index entries corresponding to each target keyword in the inverted index table, and obtain the target document identifier from the intersection of all candidate inverted index entries.
3. The document search method according to claim 2, characterized in that, The document identifiers in the inverted index entries are labeled with the first weight of the specified keyword in the documents corresponding to the document identifiers, and have been sorted according to the labeled first weights. After obtaining the target document identifier from the target inverted index entries, the method further includes: The target document is obtained from the document set according to the target document identifier, the target documents are sorted according to the order of each target document identifier in the target inverted index, and the sorted target documents are returned to the client.
4. The document search method according to claim 1, characterized in that, Before receiving the search token sent by the client, it also includes: The document update token sent by the data owner is received; the document update token includes a document identifier to be updated, a document to be updated, a keyword sequence to be updated, and a document vector to be updated; the keyword sequence to be updated contains all the keywords in the document to be updated; the document vector to be updated is constructed based on a document vector template and contains the first weight of all the keywords in the document to be updated in the document to be updated. Save the document identifier to be updated and the document to be updated to the document collection; Create a new forward index entry in the forward index table, and save the document identifier to be updated, the keyword sequence to be updated, and the document vector to be updated into the new forward index entry; The keywords in the keyword sequence to be updated are set as the keywords to be updated, and the document identifier to be updated is added to the inverted index entry corresponding to the keyword to be updated in the inverted index table.
5. The document search method according to claim 4, characterized in that, Also includes: Receive the document identifier to be deleted sent by the data owner; Based on the document identifier to be deleted, determine the corresponding forward index entry to be deleted for the document to be deleted in the forward index table; Set the inverted index entry corresponding to the keyword in the forward index entry to be deleted in the inverted index table as the inverted index entry to be updated, and remove the document identifier to be deleted from the inverted index entry to be updated; Delete the forward index entry to be deleted, and delete the document corresponding to the document identifier to be deleted from the document set.
6. The document search method according to any one of claims 1 to 5, characterized in that, The documents in the document collection have been encrypted using a first method; the target keywords in the search token and the keywords in the inverted index table have been encrypted using a second method; and the first weight in the forward index table has been encrypted using a third method.
7. A document search device, characterized in that, Applied to the server side, the device includes: The inverted search module is used to extract target keywords from a search token sent by a client when it receives such a token, and to use an inverted index table to find the target document identifier corresponding to the target document containing the target keyword. The inverted index table uses the specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the index content. The search token contains a search vector, which is constructed based on a document vector template and contains a second weight of the target keyword. The initial value of each position in the document vector template is zero, and each position corresponds to each keyword in all documents. When generating the search vector, the position of the target keyword in the document vector template is found, and the second weight of the target keyword is written into the position. The forward search module is used to determine the target first weight corresponding to the target keyword in the target document using a forward index table, and to determine the relevance value of each target document using the target first weight. The forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keyword contained in the specified document in the specified document as the index content. The forward index entries store the document identifier and the document vector. The document vector is constructed based on the document vector template and contains the first weight of all keywords in the document to be updated in the document to be updated. When generating the document vector, the position of the keyword in the document is found in the document vector template, and the first weight of the keyword in the document is written in the position. The search results output module is used to obtain the target document from the document collection according to the target document identifier, sort the target documents according to the relevance value of each target document, and return the sorted target documents to the client; The first weight in the document vector is the TF value of the corresponding keyword in the corresponding document. The data owner is responsible for calculating the TF value. The document vector is updated by the data owner to the server. The second weight in the search vector is the IDF value of the corresponding target keyword in all documents. The IDF value is updated by the data owner and periodically pushed to the client. The forward search module includes: The relevance calculation unit is used to calculate the inner product between the search vector and the document vector contained in the forward index of each target document, so as to calculate the TF-IDF value of each target keyword in each target document, and sum the TF-IDF values of each target keyword in the same target document to obtain the relevance value of each target document. The data owner and the client have plaintext keywords, while the server only has encrypted keywords. The relevant data in the document collection, the forward index table, and the inverted index table are encrypted.
8. A document search system, characterized in that, include: Server and client, among which, The server, upon receiving a search token sent by the client, extracts target keywords from the search token and uses an inverted index table to find the target document identifier corresponding to the target document containing the target keyword. The inverted index table uses a specified keyword as the key and the document identifier corresponding to the document containing the specified keyword as the index content. The search token contains a search vector constructed based on a document vector template and includes a second weight for the target keyword. Each position in the document vector template has an initial value of zero, and each position corresponds to each keyword in all documents. When generating the search vector, the server finds the position of the target keyword in the document vector template and writes the second weight of the target keyword into that position. A forward index table is used to determine the target keyword in the target document. The target first weight is determined, and the relevance value of each target document is determined using the target first weight. The forward index entries in the forward index table use the document identifier of the specified document as the key and the first weight of the keywords contained in the specified document in the specified document as the index content. The forward index entries store the document identifier and the document vector. The document vector is constructed based on the document vector template and contains the first weight of all keywords in the document to be updated in the document to be updated. When generating the document vector, the position of the keyword in the document is found in the document vector template, and the first weight of the keyword in the document is written in the position. The target document is obtained from the document set according to the target document identifier, the target documents are sorted according to the relevance value of each target document, and the sorted target documents are returned to the client. The first weight in the document vector is the TF value of the corresponding keyword in the corresponding document. The data owner is responsible for calculating the TF value. The document vector is updated by the data owner to the server. The second weight in the search vector is the IDF value of the corresponding target keyword in all documents. The IDF value is updated by the data owner and periodically pushed to the client. The server is also used to calculate the inner product between the search vector and the document vectors contained in the forward index of each target document to calculate the TF-IDF value of each target keyword in each target document. The server also sums the TF-IDF values of each target keyword in the same target document to obtain the relevance value of each target document. The client is configured to send the search token to the server and receive the sorted target document returned by the server. The data owner and the client have plaintext keywords, while the server only has encrypted keywords. The relevant data in the document collection, the forward index table, and the inverted index table are encrypted.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the document search method as described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the document search method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Retrieval method, method and device for establishing index and retrieval system
CN101819578A
Encrypt data searching method based on 5g communication standard
CN106961427A
Document searching method and device based on database
CN111522905A