Index construction and retrieval method and device supporting hidden query

Abstract

The application discloses an index construction and retrieval method and device supporting concealed query, and the method comprises the following steps: constructing an index and data storage structure containing a data table, a secondary index and a primary index; and performing concealed query retrieval based on the constructed index and data storage structure. The above method constructs an independent, general index structure supporting concealed query, and the structure can be used for efficiently performing concealed query on a keyword. In the aspect of index construction, the independent index structure is designed, and is decoupled from the concealed query technology; and the two-level index construction is used to support bucketing and positioning. In the aspect of retrieval, the primary index is used to position an index bucket; and the secondary index is used to position data. The method fills the blank of independent index in the field of concealed query, and provides a general and efficient index construction and retrieval scheme for the existing concealed query technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of privacy security computing, in particular to an index construction and retrieval method and device supporting anonymous query. BACKGROUND

[0002] In recent years, with the massive generation and processing of data, the storage and protection of data have become particularly important. In order to protect data privacy and security, various technologies have been developed and improved. Among them, the anonymous query technology (Private Information Retrieval, PIR) tends to mature, which can realize that the querying party queries data from the data providing party without exposing the content of the query. At present, there are various implementation schemes based on inadvertent transmission, homomorphic encryption, private intersection seeking, etc.

[0003] Chinese patent document CN115905238A discloses an anonymous query method, device, electronic equipment and storage medium. The method comprises: in response to a query request for to-be-queried data, determining a query identifier of the to-be-queried data; the query identifier is used to determine a sharded storage area of the to-be-queried data in a service party; sending the query identifier to the service party, for instructing the service party to query and feed back a ciphertext sequence and a corresponding position index sequence in a sharded storage area corresponding to the query identifier according to the query identifier; wherein the sharded storage area is an area in which the service party shards and stores each data in the database according to the original position index of each data; obtaining the ciphertext sequence and the position index sequence, and determining a target ciphertext from the ciphertext sequence according to the position index sequence; obtaining an anonymous key sequence sent by the service party based on an inadvertent transmission protocol, and determining a target key of the target ciphertext from the anonymous key sequence; decrypting the target ciphertext according to the target key to obtain plaintext information of the to-be-queried data.

[0004] Although the above method mentions the concept of position index, the use of the index returns the index sequence of the entire storage slice area for each data query. If multiple data not in the same storage slice area are queried, a large amount of redundant index content will be transmitted, which is low in efficiency. In the method, the querying party can determine the original index of the to-be-queried data in the database according to the unique identifier, and the scheme adopted in the embodiment is that the querying party stores all the original data indexes of the service party, which is costly in actual application. In addition, in the retrieval aspect, the content obtained by the querying party includes the ciphertext sequence, the position index sequence and the key of multiple data. The key and the ciphertext sequence can be used to solve the multiple data in the ciphertext sequence, which has a security risk.

[0005] In addition, the Chinese patent document CN116028948A discloses a keyword privacy information retrieval method and device. The method is as follows: receiving a user inputted keyword to be searched for retrieving privacy information and a first number M of obfuscated words for obfuscating the keyword to be searched, receiving a total amount X of data sent by a server; grouping the total amount X of data according to the first number to obtain a second number N; obtaining target screening information according to the second number N and the keyword to be searched; sending the second number N and the target screening information to the server, receiving a target index list fed back by the server, the target index list including Y ciphertext indexes obtained by substituting the second number N and the target screening information into a preset screening condition; obtaining a sub-data set containing privacy information corresponding to the target index list from the server according to the target index list, and obtaining privacy information corresponding to the keyword to be searched from the sub-data set. Although the method can reduce the computing and communication overheads of the electronic device to some extent and protect the user's retrieval conditions and retrieval results, the method still relies on communication redundancy to ensure its anonymity. In the index query stage, M-1 redundant encrypted index information is transmitted for each keyword searched, and there is a problem of a large amount of redundant index transmission for multiple keywords. Moreover, the index structure of the above two schemes is a unified whole with data and algorithms, and the purpose of both is not to construct an independent index supporting anonymous query.

[0006] Therefore, there is an urgent need for an independent anonymous query index construction and retrieval method supporting multiple technical solutions. SUMMARY

[0007] In view of the above, the purpose of the present application is to provide an index construction and retrieval method and device supporting anonymous query to fill the gap of independent index in the field of anonymous query and provide a general and efficient index construction and retrieval scheme for existing anonymous query technology.

[0008] In a first aspect, the present application provides an index construction and retrieval method supporting anonymous query, comprising the following steps:

[0009] An index and data storage structure containing a data table, a secondary index, and a primary index is constructed, wherein the data table includes a plurality of data buckets with unique identifiers, each data bucket containing a plurality of original data records, and the storage mode of the data bucket is compatible with the anonymous query algorithm for row query; the secondary index includes a plurality of secondary index buckets, each secondary index bucket containing a plurality of row secondary index records, each secondary index record including a keyword in the data bucket, a data bucket number corresponding to the keyword, and a row number in the data bucket with the corresponding bucket number; the primary index includes a plurality of row primary index records, each primary index record including a starting keyword, an ending keyword, a secondary index bucket number, and a secondary index bucket length;

[0010] The index and data storage structure are used for performing the anonymous query retrieval, including: determining the bucket number and length of the secondary index bucket corresponding to the keyword in the primary index based on the input keyword, and calculating the row number of the secondary index bucket based on the keyword and the length of the secondary index bucket; and obtaining the corresponding original data record in the data table based on the bucket number and the row number of the secondary index bucket and in combination with the anonymous query technology.

[0011] According to the embodiment of the present application, the construction process of the data table comprises:

[0012] The original data file is split into a plurality of data files according to the indistinguishability, and the data files are combined into a data table, and each data file is used as a data bucket in the data table, wherein the original data records in the original data file are stored in a natural storage order.

[0013] According to the embodiment of the present application, the construction process of the secondary index comprises:

[0014] The keyword of each original data record in the data bucket and the bucket number and the row number of the data bucket corresponding to the keyword are extracted as a secondary index record; the secondary index records are sorted and merged according to the keyword; the secondary index records are split into a plurality of secondary index files according to the indistinguishability, and each secondary index file is used as a secondary index bucket; and each secondary index record is assigned a storage row position in the secondary index bucket based on the keyword.

[0015] Further, the sorting and merging of the secondary index records according to the keyword comprises:

[0016] The sorting and merging of the secondary index records according to the keyword are performed by using a quick sorting and file-level merge sorting combined algorithm, and the quick sorting and file-level merge sorting combined algorithm comprises: dividing all the secondary index records into N blocks of secondary index records, wherein N is less than the number of secondary index records; and performing the following operations on each block of secondary index records: reading into the memory for quick sorting, writing the secondary index record block sorted by quick sorting into an intermediate file, and performing merge sorting on the intermediate file.

[0017] Further, the assignment of the storage row position of each secondary index record in the secondary index bucket based on the keyword comprises:

[0018] The storage row position of each secondary index record in the secondary index bucket is assigned by using a secondary index position generation algorithm, the secondary index position generation algorithm takes the keyword as input, divides the secondary index file corresponding to each secondary index bucket into two hash table spaces, uses a first hash function and a second hash function, and combines a cuckoo hash algorithm to assign the storage row position of each secondary index record in the secondary index bucket.

[0019] According to the embodiment of the present application, the construction process of the primary index comprises:

[0020] An empty primary index file is created, and the primary index comprises a primary index file, and each primary index file comprises a plurality of primary index records;

[0021] Each primary index record is constructed to comprise: each primary index record corresponds to a secondary index bucket, the minimum key in the secondary index bucket is extracted as the starting key of the primary index record, the maximum key in the secondary index bucket is extracted as the ending key of the primary index record, the bucket number of the secondary index bucket is extracted as the secondary index bucket number of the primary index record, and the secondary index bucket length is calculated according to the number and load factor of the secondary index records in the secondary index bucket as the secondary index bucket length of the primary index record.

[0022] Further, the row number of the secondary index bucket is calculated based on the key and the length of the secondary index bucket, comprising:

[0023] The row number of the secondary index bucket is calculated by using a secondary index positioning algorithm, the secondary index positioning algorithm takes the key as input, determines the hash table space size through the length of the secondary index bucket, uses a first hash function and a second hash function, and calculates the row number of the corresponding secondary index bucket by combining a cuckoo hash algorithm.

[0024] In a second aspect, the embodiment of the present application further provides an index construction and retrieval device supporting an anonymous query, comprising: an index construction unit and an anonymous retrieval unit.

[0025] The index construction unit is configured to construct an index and data storage structure comprising a data table, a secondary index and a primary index, wherein the data table comprises a plurality of data buckets with unique identifiers, each data bucket comprises a plurality of original data records, and the storage mode of the data bucket is compatible with an anonymous query algorithm for row query; the secondary index comprises a plurality of secondary index buckets, each secondary index bucket comprises a plurality of rows of secondary index records, each secondary index record comprises a key in the data bucket, a data bucket bucket number corresponding to the key and a row number in the data bucket corresponding to the bucket number; and the primary index comprises a plurality of rows of primary index records, each primary index record comprises a starting key, an ending key, a secondary index bucket number and a secondary index bucket length.

[0026] The anonymous query retrieval of the anonymous retrieval unit is performed based on the index and data storage structure constructed by the index construction unit, the bucket number and the length of the secondary index bucket corresponding to the key are determined in the primary index based on the input key, the row number of the secondary index bucket is calculated based on the key and the length of the secondary index bucket, and the corresponding original data record in the data table is obtained based on the bucket number and the row number of the secondary index bucket and in combination with the anonymous query technology.

[0027] In a third aspect, an embodiment of the present application further provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory stores a tagged ornament, and the processor implements the steps of the above-mentioned index construction and retrieval method supporting anonymous query when executing the computer program.

[0028] In a fourth aspect, an embodiment of the present application further provides a computer readable medium, wherein a computer program is stored in the computer readable medium, and the computer program implements the steps of the above-mentioned index construction and retrieval method supporting anonymous query when executed by a processor.

[0029] Compared with the prior art, the present application has at least the following beneficial effects:

[0030] The present application proposes an independent and universal index and data storage structure, which is decoupled from the anonymous query technology and can be used with any anonymous query technology supporting row retrieval, thus having better universality.

[0031] The retrieval method of the present application is efficient and the communication content is concise. On the basis of supporting anonymous query, the index is used to quickly locate the keyword to the specific data record, and the communication process of the anonymous query only involves the target data, the communication redundancy is small, and a large number of queries are supported.

[0032] The present application defines an explicit and independent index construction process, which is not affected by the anonymous query technology. DETAILED DESCRIPTION

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0034] Figure 1 is a flowchart of an index construction and retrieval method supporting anonymous query provided by an embodiment;

[0035] Figure 2 is an index and data storage structure diagram provided by an embodiment;

[0036] Figure 3 is a whole index construction flowchart provided by an embodiment;

[0037] Figure 4 is a retrieval timing diagram of an index construction and retrieval method supporting anonymous query provided by an embodiment;

[0038] Figure 5 is a structural block diagram of an index construction and retrieval device supporting a privacy query provided by an embodiment;

[0039] Figure 6 is a structural schematic diagram of a computing device provided by an embodiment. DETAILED DESCRIPTION

[0040] For the purposes of the present invention, the technical solutions and advantages thereof are more clearly apparent, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention, and do not limit the scope of protection of the present invention.

[0041] In the present embodiment, the basic concepts used include:

[0042] The privacy query client (hereinafter referred to as the client) refers to a party initiating a data query, and the purpose is to obtain data from the server so that the server cannot obtain the information of the data being queried.

[0043] The privacy query server (hereinafter referred to as the server) refers to the data provider, i.e., the party being queried, which provides privacy query services to specific clients.

[0044] Indistinguishability: The data bucket capacity represented by an integer characterizes the degree of protection of the data during privacy query. For example, if the queried party has 1000 pieces of data and the privacy indistinguishability is 100, then the data can be divided into 10 buckets, each with a capacity of 100. When querying a piece of data, the queried party can know which bucket the data is distributed in, but cannot know which specific piece of data in the bucket. Therefore, the larger the indistinguishability, the better the privacy, and the worse the query performance.

[0045] Load factor: also called load factor, generally used as the threshold for expanding the hash table. When the number of elements in the hash table and the proportion of the hash table space exceed the load factor, the hash table is expanded. In the present solution, since the hash table is constructed according to the given elements, the space size of the hash table is calculated by the number of elements and the load factor, i.e., the number of elements / load factor = hash table space.

[0046] The privacy query technology (PIR, Private Information Retrieval) selected in the present embodiment is based on a homomorphic encryption algorithm. The technology does not require key transmission during the query process. The client encrypts the row number using its own key and sends the ciphertext to the server. The server uses the data and the ciphertext to perform calculations and returns encrypted data containing only the specific row being queried. The client uses its own key to decrypt and obtain the plaintext.

[0047] AsFigure 1 As shown, the embodiment provides an index construction and retrieval method supporting anonymous query, comprising:

[0048] S110, constructing an index and data storage structure.

[0049] In the embodiment, as shown, Figure 2 The constructed index and data storage structure comprises an index and data storage structure comprising a data table, a secondary index and a primary index.

[0050] In the embodiment, the main role of the data table is to convert the original data file into a data table.

[0051] The data table is composed of multiple data files, each data file serving as a data bucket, and each data bucket having a unique numerical identifier. The data file is the basic unit of data storage. In addition to storing data, the data bucket can be used to balance computing resources and adjust the indistinguishability of data.

[0052] The data file contains multiple rows of original data records, which are stored in the natural storage order. The storage method of the data file is compatible with the anonymous query algorithm for row query, and the query technology can be used to retrieve any data file. According to the set indistinguishability, the original data records are split into multiple data files, and a data table is formed, each data file serving as a separate data bucket in the table.

[0053] For example, if the data file is a CSV format file, the first line is the table header, and each line from the second line is a structured original data record. The original data records are stored in the natural storage order. The storage method of the data file is compatible with the anonymous query algorithm, and the anonymous query technology can be used to retrieve any data file. A data table is formed by combining multiple data files with the same structure and related content. In the data table, each data file is a data bucket, each data bucket has a unique numerical identifier, and a data table composed of n data buckets has bucket numbers from 0 to n-1.

[0054] Specifically, the construction process of the data table includes: first, preserving the original data file format, directly traversing and reading the original data file, when the number of read lines reaches the indistinguishability, writing the read part to a new data file and assigning the bucket number of the data bucket; then emptying the read content and continuing to traverse the original data file until the data table is constructed.

[0055] In the embodiment, the secondary index is composed of multiple secondary index files, each secondary index file serving as a secondary index bucket, and its bucket number and identifier are divided according to the indistinguishability. Each secondary index file is composed of multiple secondary index records, which are stored in a specific order.

[0056] Each secondary index record represents the storage location of one or more data records corresponding to a certain keyword, wherein each secondary index record includes the keyword in the data bucket, the data bucket number corresponding to the keyword, and the row number in the data bucket corresponding to the bucket number. For example, the secondary index record format corresponding to one keyword 1 and two data buckets is: keyword 1, data bucket number 1-row number 1, data bucket number 2-row number 2. Wherein the keyword is a search keyword represented by an integer, and the part after the keyword represents the data bucket number and the row number of the data record related to the keyword, i.e. the location of the data record.

[0057] The storage location of each secondary index record in the file is determined by the secondary index location generation algorithm. Therefore, there can be 1, 2 or no records (represented as None) in a row in the secondary index file, and the storage mode of the secondary index record is compatible with the implicit query algorithm, and the secondary index can be searched using the implicit query technology.

[0058] Specifically, as shown in Figure 3 After the data table is constructed, the secondary index construction process includes three sub-processes executed in turn, which are the whole index record file construction, the index bucket division, and the secondary index file generation three sub-processes. The index bucket division is closely related to the primary index construction.

[0059] The whole index record file construction process is the process of generating ordered secondary index records. Specifically, for each raw data record of the original data file, first, the keyword extraction is performed, the extraction method is to obtain the field used for searching from the raw data record, and then the field is numerically processed, and then the data bucket number corresponding to the raw data record and the row number in the data bucket corresponding to the bucket number are obtained according to the information of the data table construction.

[0060] When all the secondary index records are generated, the keywords in the secondary index records are sorted and merged. In order to prevent memory overflow caused by too many records, the sorting process adopts an algorithm combining quicksort and file-level merge sort. First, the secondary index record file is divided into N blocks of secondary index records, wherein N is less than the number of secondary index records. Secondly, each block of secondary index record is read into the memory for quicksort, and the sorting result is written into an intermediate file. Finally, the intermediate file is sorted by merging, and finally an ordered secondary index record file is formed. The keyword merging is to merge the records with repeated keywords in the secondary index records into one record. In this embodiment, this process is integrated into the sorting operation.

[0061] The index bucket division is the process of dividing the ordered secondary index record file according to the setting of the secondary index bucket capacity. The implementation process is the same as the data bucket division, which is divided according to the indistinguishability. The capacity of the secondary index bucket should be not less than the implicit indistinguishability.

[0062] For the generation of the secondary index file, for each secondary index record in each secondary index bucket, a secondary index location generation algorithm is used to determine its storage location. The secondary index location generation algorithm: first, the secondary index file corresponding to each secondary index bucket is divided into two hash table spaces according to a set load factor, the embodiment selects a load factor of 1.0, MurmurHash3, crc32 as a hash function, and combines a Cuckoo hashing algorithm to assign a storage row location in the bucket for each secondary index record.

[0063] When performing the allocation, although the Cuckoo hashing algorithm can effectively reduce the collision probability, collisions will still inevitably occur. A collision processing strategy is also adopted, by directly storing the collided multiple records in the same row. When the secondary index record location allocation in all buckets is completed and written to the file, the secondary index construction is completed.

[0064] In the embodiment, the primary index is composed of a single file, i.e., a primary index file, which contains multiple rows of primary index records. The primary index is mainly used to determine the location of the secondary index, and its volume is relatively small, which is directly provided to the client by the server during retrieval. The format of each row of the primary index record is: start key, end key, secondary index bucket number, and secondary index file length. The start key is less than or equal to the end key, which represents the key range corresponding to the primary index record, and the secondary index bucket number represents the secondary index bucket number where the key range is located, and the secondary index file length information stores the length of the two hash tables in the index bucket, which is used to provide parameters for the secondary index retrieval process.

[0065] When the secondary index is constructed to the bucketing stage, the primary index can be constructed at the same time. Specifically, first, an empty primary index file is created, and then the following operations are performed on the ordered records of each secondary index bucket: 1. Extract the minimum key in the secondary index record as the start key. 2. Extract the maximum key in the secondary index record as the end key. 3. Extract the number of the secondary index bucket. 4. According to the number of secondary index records in the secondary index bucket and the load factor, the length information of the secondary index file is calculated. 5. The information of 1-4 is appended to the primary index file as a primary index record. When the primary index record information generated by all secondary index buckets is written to the primary index file, the primary index construction is completed.

[0066] At this point, a data table containing several data buckets, secondary indexes containing several buckets, and a primary index file have been generated. This process builds the key components of the hidden query index, including the data table, secondary indexes, and primary indexes. This process relies on the results of processing the original data, so it is executed after the data table is built. Once this process is complete, keywords combined with indexes can be used to perform efficient hidden queries on the original data records.

[0067] S120 , performing hidden query retrieval based on the constructed index and data storage structure.

[0068] The process of using S101 to build secondary indexes, primary indexes, and data tables for hidden retrieval starts with the client entering query keywords and ends with obtaining relevant raw data.

[0069] The timing of the entire retrieval process, such as Figure 4 As shown in Figure 1, the process is divided into three stages: primary index retrieval, secondary index retrieval, and data retrieval. Primary index retrieval occurs on the client side and does not require the use of hidden query technology. Secondary index retrieval and data retrieval involve communication of the retrieval content and require the use of hidden query technology to retrieve the index and data.

[0070] The first-level index retrieval is the process of retrieving the first-level index file using keywords as input to obtain the corresponding second-level index bucket information. The first-level index file is obtained, and the bucket number and length of the second-level index bucket corresponding to the keyword are determined based on the input keyword. Specifically, first, before starting to use the first-level index for retrieval, the client needs to obtain the first-level index file, which is transmitted and provided by the server. Secondly, after obtaining the first-level index file, the client loads the first-level index file into memory, enters the keyword, and uses the binary search method to locate the first-level index record in the first-level index. Finally, when the keyword range in the first-level index record matches the input keyword, the bucket number and file length information of the second-level index can be obtained based on the information in the record.

[0071] After the primary index search is complete, the bucket number corresponding to the secondary index to be searched has been determined. According to the definition of indistinguishability, only the data within the bucket needs to be kept anonymous, so the bucket number does not need to be anonymized.

[0072] The row number of the secondary index bucket is calculated based on the keyword and the length of the secondary index bucket. The specific process is as follows:

[0073] Firstly, the secondary index locating algorithm is used to calculate the line number of the index record to be searched in the secondary index file according to the query keyword. The main process of the secondary index locating algorithm is as follows: 1. The input keyword is taken as the input, and the same two hash functions (MurmurHash3, crc32) as the secondary index position generating algorithm in the secondary index construction are selected as the first hash function and the second hash function. 2. The size of the hash table space is determined according to the length of the secondary index bucket found by the primary index record, and in this embodiment, the length of the secondary index bucket is the sum of the two hash table spaces. 3. The two line numbers of the corresponding secondary index bucket are calculated for the keyword by using the cuckoo hashing algorithm in combination with the hash function and the size of the hash table space.

[0074] Secondly, the client sends the bucket number to the server and uses the anonymous query technology to search the line number of the secondary index bucket in the secondary index file determined by the server, to obtain two secondary index records, and in this process, the server can only know the bucket to be queried and cannot know which data in the bucket is queried.

[0075] Finally, the secondary index record returned contains the information of the keyword, and the client can exclude the redundant secondary index record according to the search keyword, because only one of the two line numbers calculated by the cuckoo hashing is related to the keyword.

[0076] On the basis of obtaining the bucket number and the line number of the secondary index bucket, the corresponding original data record in the data table is obtained based on the bucket number and the line number of the secondary index bucket in combination with the anonymous query technology. The specific process is as follows: firstly, according to the secondary index record, the bucket number and the line number of the data record corresponding to the keyword in the data table can be obtained. Secondly, the client sends the bucket number to the server in plaintext. Finally, the data in the specific data bucket of the server is searched by using the anonymous query technology to obtain the corresponding data record. The anonymous query technology used can meet the requirement of supporting the anonymous query of data with the line number as the input.

[0077] The method provided in this embodiment is efficient in retrieval and concise in communication content. In terms of retrieval, the binary method with a time complexity of O(log(n)) is used for the retrieval of the record in the primary index file by the primary index retrieval, and the number of records is only related to the number of secondary index buckets; the cuckoo hashing is used to determine the record line number in the secondary index retrieval process, and the time complexity is O(1); the data line number is directly determined by the result returned by the secondary index retrieval in the data retrieval process, and the time complexity is O(1); therefore, the overall retrieval method is relatively efficient.

[0078] In terms of communication, the first-level index retrieval process needs to transmit a first-level index file storing second-level index bucket information, the overall volume is small, and even if multiple rounds of keyword retrieval are performed, the file only needs to be transmitted once; the second-level index retrieval process can support returning content with a redundancy of 1 times the query content, that is, when performing n times of keyword query, the number of returned second-level index records is 2n, the redundancy is n, and the redundancy is irrelevant to the indistinguishability; in the data retrieval process, each query is accurate to a specific bucket number and row number, and can support non-redundant communication. Therefore, using the scheme has high communication efficiency.

[0079] In addition, since the embodiment scheme is an independent index, it needs to be used in combination with a specific obfuscated query technology during retrieval, therefore, the retrieval efficiency and communication amount in actual application are also affected by the selected obfuscated query technology.

[0080] Based on the same inventive concept, the embodiment also provides an index construction and retrieval device 500 supporting obfuscated query, as shown in Figure 5 The index construction and retrieval device 500 includes an index construction unit 510 and an obfuscated retrieval unit 520.

[0081] The index construction unit 510 is configured to construct an index and data storage structure including a data table, a second-level index, and a first-level index, wherein the data table includes a plurality of data buckets with unique identifiers, each data bucket contains a plurality of original data records, and the storage mode of the data bucket is compatible with an obfuscated query algorithm for row query; the second-level index includes a plurality of second-level index buckets, each second-level index bucket contains a plurality of rows of second-level index records, each second-level index record includes a keyword in the data bucket, a data bucket number corresponding to the keyword, and a row number in the data bucket corresponding to the bucket number; and the first-level index includes a first-level index file, each first-level index file includes a plurality of rows of first-level index records, each first-level index record includes a start keyword, an end keyword, a second-level index bucket number, and a second-level index bucket length.

[0082] The obfuscated retrieval unit 520 is configured to perform obfuscated query retrieval based on the index and data storage structure constructed by the index construction unit, including: obtaining a first-level index file, and determining a bucket number and a length of a second-level index bucket corresponding to an input keyword; calculating a row number of the second-level index bucket based on the keyword and the length of the second-level index bucket; and obtaining corresponding original data records in the data table based on the bucket number and the row number of the second-level index bucket in combination with an obfuscated query technology.

[0083] It should be noted that the index construction and retrieval device supporting anonymous query provided in the above embodiment should be divided into the above functional modules when performing index construction and anonymous retrieval, and the above functions can be completed by different functional modules according to needs, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the index construction and retrieval device supporting anonymous query provided in the above embodiment and the index construction and retrieval method supporting anonymous query embodiment belong to the same concept, and the specific implementation process is described in detail in the index construction and retrieval method supporting anonymous query embodiment, which will not be repeated here.

[0084] Based on the same inventive concept, the embodiment also provides a computing device, as shown in the figure, which includes a processor, an internal bus, a network interface, a memory, and a storage, and of course can also include other hardware required by the business. The processor reads the corresponding computer program from the storage into the memory and then runs to implement the above-mentioned index construction and retrieval method supporting anonymous query, including: Figure 6

[0085] S110, constructing an index and a data storage structure.

[0086] S120, performing anonymous query retrieval based on the constructed index and data storage structure.

[0087] Among them, the storage can be a volatile storage in the near end, such as RAM, and can also be a non-volatile storage, such as ROM, FLASH, floppy disk, mechanical hard disk, etc., and can also be a remote storage cloud. The processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), that is, the steps of the index construction and retrieval method supporting anonymous query can be implemented by these processors.

[0088] Based on the same inventive concept, the embodiment also provides a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the above-mentioned index construction and retrieval method supporting anonymous query, including:

[0089] S110, constructing an index and a data storage structure.

[0090] S120, performing anonymous query retrieval based on the constructed index and data storage structure.

[0091] Among them, the computer readable storage medium can be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0092] ​The above detailed description of the specific embodiments of the present application has described the technical solutions and beneficial effects of the present application, and it should be understood that the above description is only the most preferred embodiment of the present application and is not intended to limit the present application. Any modifications, supplements and equivalent replacements made within the principle range of the present application shall be included in the protection range of the present application.

Claims

1. A method for index construction and retrieval supporting hidden queries, characterized in that: The following steps are involved: Construct an index and data storage structure including a data table, a secondary index, and a primary index, wherein the data table includes multiple data buckets with unique identifiers, each data bucket contains multiple original data records, and the storage method of the data buckets is compatible with the hidden query algorithm for row-by-row queries; the secondary index includes multiple secondary index buckets, each secondary index bucket contains several rows of secondary index records, and each secondary index record includes a keyword in the data bucket, the data bucket number corresponding to the keyword, and the row number in the data bucket with the corresponding bucket number; the primary index includes a primary index file, each primary index file contains several rows of primary index records, and each primary index record includes a starting keyword, an ending keyword, a secondary index bucket number, and a secondary index bucket length; Hidden query retrieval is performed based on the constructed index and data storage structure, including: obtaining the first-level index file, and determining the bucket number and length of the secondary index bucket corresponding to the keyword based on the input keyword; calculating the row number of the secondary index bucket based on the keyword and the length of the secondary index bucket; based on the bucket number and row number of the secondary index bucket, combining hidden query technology to obtain the corresponding original data records in the data table.

2. The index construction and retrieval method supporting hidden queries according to claim 1, characterized in that: The process of constructing the data table includes: The original data file is split into several data files according to the indistinguishability to form a data table. Each data file serves as a data bucket in the data table. The original data records in the original data file are stored in the natural storage order.

3. The index construction and retrieval method supporting hidden queries according to claim 1, characterized in that: The process of building the secondary index includes: Extract the keyword of each original data record in the data bucket, the data bucket number corresponding to the keyword, and the row number in the data bucket with the corresponding bucket number as a secondary index record; sort and merge all secondary index records by keyword; split all secondary index records into several secondary index files according to indistinguishability, and regard each secondary index file as a secondary index bucket; assign a storage row position in the secondary index bucket to each secondary index record based on the keyword.

4. The index construction and retrieval method supporting hidden queries according to claim 3 is characterized in that: The keyword sorting and keyword merging of all secondary index records includes: An algorithm combining quick sort and file-level merge sort is used to perform keyword sorting and keyword merging on all secondary index records. The algorithm combining quick sort and file-level merge sort includes: dividing all secondary index records into N blocks of secondary index record blocks, where N is less than the number of secondary index records; performing the following operations on each block of secondary index record blocks: reading into memory for quick sorting, writing the quickly sorted secondary index record blocks into an intermediate file, and merge sorting the intermediate file.

5. The index construction and retrieval method supporting hidden queries according to claim 3, characterized in that: The step of allocating a storage row position in a secondary index bucket for each secondary index record based on the keyword includes: A secondary index position generation algorithm is used to assign a position in a secondary index bucket to each secondary index record. The secondary index position generation algorithm takes a keyword as input, divides the secondary index file corresponding to each secondary index bucket into two hash table spaces, uses a first hash function and a second hash function, and combines the cuckoo hash algorithm to assign a storage row position in the secondary index bucket to each secondary index record.

6. The index construction and retrieval method supporting hidden queries according to claim 1, characterized in that: The process of constructing the primary index includes: Create an empty first-level index file. The first-level index contains a first-level index file. Each first-level index file contains several first-level index records. The construction of each first-level index record includes: each first-level index record corresponds to a second-level index bucket, extracting the minimum keyword in the second-level index bucket as the starting keyword of the first-level index record, extracting the maximum keyword in the second-level index bucket as the ending keyword of the first-level index record, extracting the bucket number of the second-level index bucket as the second-level index bucket number of the first-level index record, and calculating the second-level index bucket length according to the number of second-level index records in the second-level index bucket and the load factor as the second-level index bucket length of the first-level index record.

7. The index construction and retrieval method supporting hidden queries according to claim 5, characterized in that: Calculating the row number of the secondary index bucket based on the keyword and the length of the secondary index bucket includes: A secondary index positioning algorithm is used to calculate the row number of the secondary index bucket. The secondary index positioning algorithm takes the keyword as input, determines the size of the hash table space by the length of the secondary index bucket, uses the first hash function and the second hash function, and combines the cuckoo hash algorithm to calculate the row number of the corresponding secondary index bucket.

8. An index construction and retrieval device supporting hidden queries, characterized in that: include: Index building unit, hidden retrieval unit; The index construction unit is used to construct an index and data storage structure including a data table, a secondary index, and a primary index, wherein the data table includes multiple data buckets with unique identifiers, each data bucket contains multiple original data records, and the storage method of the data buckets is compatible with the hidden query algorithm for row-by-row query; the secondary index includes multiple secondary index buckets, each secondary index bucket contains several rows of secondary index records, and each secondary index record includes a keyword in the data bucket, a data bucket number corresponding to the keyword, and a row number in the data bucket with the corresponding bucket number; the primary index includes a primary index file, each primary index file contains several rows of primary index records, and each primary index record includes a starting keyword, an ending keyword, a secondary index bucket number, and a secondary index bucket length; The hidden retrieval unit is used to perform hidden query retrieval based on the index and data storage structure constructed by the index construction unit, including: obtaining the primary index file, and determining the bucket number and length of the secondary index bucket corresponding to the keyword based on the input keyword; calculating the row number of the secondary index bucket based on the keyword and the length of the secondary index bucket; based on the bucket number and row number of the secondary index bucket, combining the hidden query technology to obtain the corresponding original data record in the data table.

9. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The memory stores accessories with labels, and the processor implements the steps of the index construction and retrieval method supporting hidden queries as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is processed and executed, the steps of the index construction and retrieval method supporting hidden queries according to any one of claims 1 to 7 are implemented.

Images