Method and system for compressed and fast encrypted similarity searching and database analysis - Patents.com

Abstract

A method (100) for encrypted similarity searching of a database includes the steps of: (i) providing (120) a database including a plurality of homomorphically encrypted fixed-length data fingerprints; (ii) receiving an encrypted query (130); (iii) generating (140) a fixed-length query data fingerprint; (iv) homomorphically encrypting (150) the fixed-length query data fingerprint; (v) generating (160) a fixed-size query data fingerprint table; and (vi) generating a homomorphically encrypted fixed-length query data fingerprint table. The method includes (170) comparing the homomorphically encrypted fixed-length query data fingerprint against each of the homomorphically encrypted fixed-length data fingerprints, where each comparison generates a distance; (vii) using the generated distances between the homomorphically encrypted fixed-length query data fingerprint and each of the homomorphically encrypted fixed-length data fingerprints to identify (180) one or more data files in the database having the smallest generated distances; and (viii) reporting (190) the identified one or more data files.

Description

[Technical field]

[0001] The present disclosure is generally directed to methods and systems for encrypted similarity searching of data. [Background technology]

[0002]

[0002] Many data applications require similarity searching. For example, in the medical field, finding patients and / or treatments with similar characteristics can provide insight into patient progression, prediction and / or prognosis, patient treatment, etc. Thus, identifying similar patients can have significant benefits to patients' healthcare outcomes. One concern with similarity searching of medical data in medical databases is that searching for similar patients may require the transmission or communication of privacy-sensitive medical information. The transmission of this sensitive information may be a regulatory issue in addition to privacy and / or security risks.

[0003]

[0003] Many electronic health record systems utilize the Fast Healthcare Interoperability Resource (FHIR) format created by the Health Level Seven International (HL7) healthcare standards consortium to store and communicate medical data. FHIR is a standard that describes data formats and elements (i.e., resources) and APIs for exchanging electronic health records. However, FHIR files can be very large files, so similarity searches using FHIR files can be a cumbersome process.

[0004]

[0004] One approach is the use of zk-SNARKs (zero-knowledge Succinct Non-interactive Arguments of Knowledge), which are cryptographic proof techniques for establishing knowledge or ownership in a manner that preserves confidentiality while minimizing the amount of bandwidth used for communication. This approach allows a party to prove properties of a computation, but it does not perform the computation itself in a fully cryptographic manner.

[0005]

[0005] Another approach for secure similarity searching of medical data is to search the data while it remains encrypted, and current methods explore the use of methods such as homomorphic encryption, which allow analysis of data while it remains encrypted. However, homomorphic solutions are severely limited in the amount of data they can process, and even the best solutions remain severely limited in real-world use. Summary of the Invention [Problem to be solved by the invention]

[0006]

[0006] Thus, there is a continuing need in the art for methods and systems that enable fast and efficient similarity searching of databases while ensuring the privacy and security of queries. [Means for solving the problem]

[0007]

[0007] The present disclosure is directed to an inventive method and system for similarity searching of a database. Various embodiments and implementations herein are directed to a method and system for querying a database using homomorphic encryption. According to an embodiment, a homomorphically encrypted fixed-length query data fingerprint is queried against a homomorphically encrypted database to identify a distance between the query and data in the database. The similarity search system comprises a database including a plurality of homomorphically encrypted fixed-length data fingerprints, each of which represents a data file in the database. The homomorphically encrypted fixed-length data fingerprints are randomly ordered in a fixed-size query database table in the database. The system receives a query to be queried against the database in a similarity search, generates a fixed-length query data fingerprint, and homomorphically encrypts the fixed-length query data fingerprint. The system generates a fixed-size query data fingerprint table that is the same size as the fixed-size database table, and then compares the fingerprints in the two tables to generate a distance between the query and each fingerprint in the database. The system identifies the files in the database that have the smallest distance to the query and reports the distance and / or the identified files.

[0008]

[0008] In general, in one aspect, a method is provided for encrypted similarity searching of a database, the method including the steps of: (i) providing a database including a plurality of homomorphically encrypted fixed-length data fingerprints, each of which represents a data file in the database, the plurality of homomorphically encrypted fixed-length data fingerprints being randomly ordered in a fixed-size database table; (ii) receiving a query to be queried against the database in a similarity search; (iii) generating a fixed-length query data fingerprint from the received query; (iv) homomorphically encrypting the fixed-length query data fingerprint; and (v) generating a fixed-size query data fingerprint table, the size of the fixed-size query data fingerprint table being a fixed-size query data fingerprint table. is equal to the size of the fixed-size database table; (vi) comparing the homomorphically encrypted fixed length query data fingerprint against each of the homomorphically encrypted fixed length data fingerprints using the fixed-size query data fingerprint table and the fixed-size database table, each comparison generating a distance; (vii) using the generated distances between the homomorphically encrypted fixed length query data fingerprint and each of the homomorphically encrypted fixed length data fingerprints to identify one or more data files in the database having the smallest generated distances; and (viii) reporting the identified one or more data files in the database.

[0009]

[0009] According to an embodiment, the method further comprises generating a plurality of homomorphically encrypted fixed length data fingerprints.

[0010]

[0010] According to an embodiment, the method further includes a step of generating a fixed-size database table including a plurality of homomorphically encrypted fixed-length data fingerprints, wherein the plurality of homomorphically encrypted fixed-length data fingerprints are randomly ordered in the fixed-size database table.

[0011]

[0011] According to an embodiment, the distance between each homomorphically encrypted fixed-length data fingerprint and the homomorphically encrypted fixed-length query data fingerprint is used to populate a distance table, which is the same size as the fixed-size query data fingerprint table and the fixed-size database table.

[0012] According to an embodiment, the distance table is homomorphically encrypted.

[0013]

[0013] According to an embodiment, the generated distances are homomorphically encrypted.

[0014]

[0014] According to an embodiment, the minimum generated distance is determined by a pre-determined setting or is user-defined.

[0015] According to an embodiment, the reporting step further comprises the generated distance between the query and the identified one or more data files in the database.

[0016]

[0016] According to an embodiment, the encrypted query is an updated version of the database, and each fixed-length data fingerprint represents one version of the database or one version of a data file within the database.

[0017]

[0017] According to an embodiment, the encrypted query is an updated version of the reference genome, and each fixed-length data fingerprint represents the reference genome.

[0018]

[0018] Another aspect is a system for encrypted similarity searching of a database, the system comprising a database including a plurality of homomorphically encrypted fixed-length data fingerprints, each of which represents a data file in the database, the plurality of homomorphically encrypted fixed-length data fingerprints being randomly ordered in a fixed-size database table, a query to be queried against the database in a similarity search, (i) generating a fixed-length query data fingerprint from the query, (ii) homomorphically encrypting the fixed-length query data fingerprint, and (iii) generating a fixed-size query data fingerprint table, the size of the fixed-size query data fingerprint table being equal to or larger than the size of the fixed-size database table. (iv) comparing the homomorphically encrypted fixed length query data fingerprint against each of the homomorphically encrypted fixed length data fingerprints using the fixed size query data fingerprint table and the fixed size database table, each comparison generating a distance; (v) using the generated distances between the homomorphically encrypted fixed length query data fingerprint and each of the homomorphically encrypted fixed length data fingerprints to identify one or more data files in the database having the smallest generated distance; and (vi) generating a report including the identified one or more data files in the database.

[0019] According to an embodiment, the system further comprises a user interface that provides the generated report.

[0020]

[0020] It should be recognized that all combinations of the above concepts and additional concepts discussed in more detail below (unless such concepts are mutually inconsistent) are contemplated as part of the inventive subject matter disclosed herein. In particular, all combinations of subject matter claimed at the end of this disclosure are contemplated as part of the inventive subject matter disclosed herein. It should also be recognized that the terminology explicitly used in this specification and that may also appear in any disclosures incorporated by reference should be given the meaning that is most consistent with the particular concepts disclosed herein.

[0021]

[0021] These and other aspects of various embodiments will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0022]

[0022] In the drawings, like reference characters generally refer to like parts throughout the drawings. The drawings illustrate features and ways of implementing various embodiments, and are not to be construed as limiting other possible embodiments falling within the scope of the appended claims. Also, the drawings are not necessarily to scale, with emphasis instead being placed generally on illustrating the principles of the various embodiments. [Brief description of the drawings]

[0023] [Figure 1] 1 is a flowchart of a method for encrypted similarity searching of a database according to an embodiment. [Diagram 2]

[0024] FIG. 1 is a schematic diagram of a similarity search system according to an embodiment. [Diagram 3]

[0025] 1 is a schematic diagram of a method for querying a database using a similarity search system, according to an embodiment. [Figure 4]

[0026] FIG. 1 is a schematic diagram of an approach using a similarity search system, according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024]

[0027] This disclosure describes various embodiments of systems and methods configured to enable searching of homomorphically encrypted files and the queries themselves to ensure privacy and security of those files. More generally, applicants have realized and recognized that it would be beneficial to provide a method and system for querying a database without revealing the contents of the query to the database owner. A similarity search system includes a database including a plurality of homomorphically encrypted fixed-length data fingerprints, each of which represents a data file in the database. The homomorphically encrypted fixed-length data fingerprints are randomly ordered in a fixed-size query database table in the database. The system receives an encrypted query to be queried against the database in a similarity search, generates a fixed-length query data fingerprint, and homomorphically encrypts the fixed-length query data fingerprint. The system generates a fixed-size query data fingerprint table that is the same size as the fixed-size database table, and then compares the fingerprints in the two tables to generate a distance between the query and each fingerprint in the database. The system identifies the files in the database that have the smallest distance to the query and reports the distance and / or the identified files.

[0025]

[0028] In an embodiment, researchers and / or healthcare professionals can utilize healthcare or research systems to quickly find similar data files, patients, or other data. One continuing challenge in dealing with real-world data and finding similar data is the size of the data. The methods and systems described or otherwise envisioned herein create homomorphically encrypted fingerprints of queries and data files, allowing for fast and secure searches. In addition to solving the data size problem faced by homomorphic encryption of real data, the methods and systems described or otherwise envisioned herein provide additional positive benefits. For example, there are many use cases for fast and secure comparisons, including determining database updates and other similarity search efforts. Thus, the methods and systems described or otherwise envisioned herein are highly beneficial in applications where both security and speed or performance are important considerations or requirements.

[0026]

[0029] The embodiments and implementations disclosed or otherwise envisioned herein may be utilized with any system, including, but not limited to, medical databases or systems. For example, one application of the embodiments and implementations herein is to improve medical monitoring or analysis systems, such as Philips® patient monitoring systems, such as Philips IntelliSpace® Precision Medicine products (manufactured by Koninklijke Philips, NV), among many other products. However, the disclosure is not limited to these devices or systems, and the disclosures and embodiments disclosed herein may encompass any device or system that utilizes or otherwise requires similarity searching or comparison.

[0027]

[0030] Referring to FIG. 1, there is shown a flow chart of a method 100 for encrypted similarity searching of a database in one embodiment. The method can be performed by a similarity searching system. It should be understood that the method described in connection with the figure is provided by way of example only and does not limit the scope of the present disclosure. The similarity searching system can be any of the systems described or otherwise envisioned herein. The similarity searching system can be a single system or a plurality of different systems.

[0028]

[0031] In step 110 of the method, a similarity search system 200 is provided. For example, referring to an embodiment of the similarity search system 200 as shown in FIG. 2, the system includes one or more of a processor 220, a memory 230, a user interface 240, a communication interface 250, and a storage 260, interconnected via one or more system buses 212. It will be understood that FIG. 2 is an abstraction in some respects, and the actual organization of the components of the system 200 may be different and more complex than that illustrated. Furthermore, the similarity search system 200 may be any of the systems described or otherwise envisioned herein. Other elements and components of the system 200 are disclosed and / or envisioned elsewhere herein.

[0029]

[0032] The method 100 using the similarity search system 200 can be realized in many ways using various types of data. For example, the method may be utilized with FHIR files, since FHIR files are the most common type of medical data for which similarity searches are required, but it may be any type of data. The goal of similarity search is to find, for a FHIR record, the most similar record in a database of FHIR records. The most straightforward solution to this problem would be to use a bit vector representation of the FHIR file fingerprint as a hash key for a hash table of homomorphically encrypted data (the original FHIR file). This could be a matter of simply creating a bit vector of a given file through fingerprinting, and then searching the hash table to find the match and the closest entry. However, such an approach has the drawback that it leaks the bit vector value of a given file to the owner of the database. To prevent this leakage, methods and systems described or otherwise envisioned herein include an approach where the system homomorphically encrypts one or more fixed-size tables containing a random ordering of all the bit vector values ​​of the FHIR files in the database and compares it against a homomorphically encrypted table containing the bit vector values ​​of the files being queried.

[0030]

[0033] In an optional step 115 of the method, a database structure for similarity searching is generated. The database structure may be generated by the similarity search system or by a separate system. The database structure includes at least a fixed-size database table populated with randomly ordered homomorphically encrypted fixed-length data fingerprints, each of which represents a data file in the database. The fixed-size database table may be generated using known methods for generating data tables. The homomorphically encrypted fixed-length data fingerprints may be generated in accordance with methods described below and herein or otherwise envisioned. Once generated, the database structure is stored in a local or remote database or system. For example, the database structure may be a component of the similarity search system, or the similarity search system may be in other forms of wired and / or wireless communication with the generated and stored database structure.

[0031]

[0034] In step 120 of the method, a database structure for similarity searching is provided. The database includes a number of data items, which may be any data to be queried or otherwise searched. For example, in one application, the database includes medical data and the data items are Fast Healthcare Interoperability Resource (FHIR) files, although many other data types and applications are possible.

[0032]

[0035] The database includes a plurality of homomorphically encrypted fixed-length data fingerprints, each of which represents a data file in the database. The homomorphically encrypted fixed-length data fingerprints are generated by methods described below and herein or otherwise envisioned. The homomorphically encrypted fixed-length data fingerprints are randomly ordered in a fixed-size database table. For example, if the database includes 100 different data items, the fixed-size database table will include 100 rows of homomorphically encrypted fixed-length data fingerprints, each row representing one of the 100 data items.

[0033]

[0036] The database may be a local or remote database or system, for example, the database may be a component of a similarity search system, or the similarity search system may be in otherwise wired and / or wireless communication with the generated and stored database structure.

[0034]

[0037] In step 130 of the method, the similarity search system receives an encrypted query that is queried against the database. The query can be any data or anything else that can be used in a search or otherwise queried against the database. For example, in one embodiment, the query includes patient data and the database includes data about a number of other patients, and the goal of the query is to identify the most similar patient among the number of other patients. As another example, the query may be an update, and the database is searched for similarities between the data in the database and / or the database itself and the update.

[0035]

[0038] The encrypted query may be received at the similarity search system, such as through a user interface. Alternatively, the query may be received from a remote system in wired and / or wireless communication with the similarity search system. Some or all of the similarity search system may be a cloud-based system. For example, the similarity search system may include a centralized database and one or more local or remote interfaces through which queries are submitted or otherwise received.

[0036]

[0039] In step 140 of the method, a fixed length query data fingerprint is generated from the received encrypted query. The fixed length query data fingerprint is generated using known methods for generating fingerprints from data. According to an embodiment, the fixed length data fingerprint comprises a fixed length, floating point valued feature vector. According to an embodiment, the fixed length, floating point valued feature vector comprises a number of pre-determined sub-feature vectors. The length of the fixed length data fingerprint is pre-determined by a user or the system. For example, the length depends on the number of data items in the database, the type of data in the database, or other parameters of the database and / or the query.

[0037]

[0040] In step 150 of the method, the fixed-length query data fingerprint is homomorphically encrypted. The fixed-length data fingerprint, such as the fingerprinted data items in the database and the fingerprinted query, is homomorphically encrypted using any suitable method for homomorphic encryption, including but not limited to methods described or otherwise envisioned herein.

[0038]

[0041] According to an embodiment, the method utilizes practical homomorphic encryption instead of true, fully homomorphic encryption to limit the amount of computation performed on the data in the method to a fixed amount. Because the method involves a known function such as genomic query or other types of query, the method can restrict the homomorphic encryption strictly to this function, and can improve performance in terms of storage and computation time by fine-tuning one or more parameters towards perfection.

[0039]

[0042] Also, when using a conventional, i.e., fully homomorphic encryption scheme, rather than practical homomorphic encryption, the encrypted data piece will contain a certain amount of noise within it that contaminates the original data. This noise will grow stronger and larger in size during homomorphic computation, eventually catching up with the original encrypted data, at which point it can no longer be correctly decrypted. Therefore, in order to perform many computations in a homomorphic encryption system, the homomorphically encrypted data must be refreshed frequently to reduce this noise, especially when the encrypted size of the encrypted data is small. It is this encrypted size and other similar aspects, including parameters, that are relevant when and how to reduce noise, that can be optimized to achieve a correspondingly performing solution for querying homomorphically encrypted genomic data.

[0040]

[0043] According to an embodiment, practical homomorphic encryption utilizes a Ring LWE-based cryptosystem that uses a 2-power cyclotomic integer ring, such as that mentioned in Lyubashevsky et al., "On ideal lattices and learning with errors over rings," JACM (2013). In this cryptosystem, raw data is encoded into what is called plaintext, which is a polynomial operating in a space that is typically a quotient ring of polynomials, as shown in Equation 1 below. Z t [x] / (x n +1) (Formula 1)

[0041]

[0044] On the other hand, the encrypted data, called the ciphertext, is a polynomial operating in a space that is typically a quotient ring of polynomials, as shown in Equation 2 below. Z q [x] / (x n +1) (Formula 2)

[0042]

[0045] In each case, n is a power of 2, t is the modulus of the plaintext ring, and q is the modulus of the ciphertext ring. Each input, whether it is an integer or a fixed-length sequence of bytes, must first be converted accordingly to the plaintext polynomial format, and then transformed into the ciphertext polynomial format after encryption.

[0043]

[0046] According to an embodiment, one factor of homomorphic encryption performance and a continuing boom in homomorphic encryption solutions is the introduction of batching. Batching is a powerful technique that allows single instruction multiple data (SIMD) operations to be performed on homomorphically encrypted data. As discussed herein, each input needs to be encoded into a plaintext polynomial, which is very wasteful since it encodes only one single integer into a plaintext polynomial that theoretically has enough space to store thousands of such integers. In practice, the n used for the quotient ring of both polynomials is almost always at least 1024 or more. In the present method, batching fills the entire polynomial space with as much information as possible and exploits certain ring isomorphisms in a way that considers both additive and multiplicative homomorphisms. Batching can provide very large performance gains in computations that may use single instruction multiple data operations, making batching one of the most powerful and important techniques in modern homomorphic encryption.

[0044]

[0047] Thus, as mentioned above, homomorphic encryption works by representing both the plaintext and the ciphertext by polynomials. When applying homomorphic encryption, it is first necessary to choose certain parameters for these polynomials. The most crucial parameter is the degree n, which is the exponent shared between both the plaintext polynomial and the ciphertext polynomial, and this determines the number of terms in both polynomials. According to one embodiment, the method utilizes n, which is equal to the maximum size of a Cuckoo hash table, 8192. However, other values ​​of n are possible.

[0045]

[0048] According to an embodiment, the method uses 3686401 as the plaintext modulus t, which is a prime number congruent to 1 modulo 2×n, so that the plaintext can be viewed as a 2×(n / 2) matrix with integer components modulo the plaintext modulus. This choice allows the system to take advantage of SIMD operations and batching techniques to greatly improve performance for large data sets such as Cuckoo hash tables. However, other values ​​of the plaintext modulus t are also possible.

[0046]

[0049] According to an embodiment, the final parameter q, the modulus of the ciphertext, is determined by the homomorphic encryption framework depending on the level of security desired to be achieved. According to an embodiment, the system can utilize 128-bit security. Higher security levels such as 256 or 512-bit security are also possible, but homomorphic encryption provides less flexibility in dealing with the inevitable noise that occurs in the calculations with increased security. Thus, although other values ​​of q are possible, the system utilizes 128-bit security. According to an embodiment, the fixed-length query data fingerprint can be generated by the methods and systems described in co-pending U.S. patent application Ser. No. XX / XXX,XXX, among others.

[0047]

[0050] Thus, the outcome of step 150 of the method is a homomorphically encrypted fixed-length query data fingerprint that can be utilized in subsequent steps of the method. The homomorphically encrypted fixed-length query data fingerprint may be utilized immediately or may be stored in local or remote storage for later use.

[0048]

[0051] As noted above, data items in the database against which the query is made can be fingerprinted and homomorphically encrypted using this same method. Multiple homomorphically encrypted fixed-length data fingerprints may be used for querying immediately or at any time in the future.

[0049]

[0052] In step 160 of the method, the system generates a fixed-size query data fingerprint table, where each row is the same homomorphically encrypted query data fingerprint. Each row of the query data fingerprint table will be compared against each row of randomly ordered homomorphically encrypted data fingerprints, where each data fingerprint represents a data file in the database. Thus, the size of the fixed-size query data fingerprint table is determined by the size of the fixed-size data fingerprint table in the database being queried. Once generated, the fixed-size query data fingerprint table may be used immediately or may be stored in local or remote storage for later querying.

[0050]

[0053] Thus, the size of the fixed size data fingerprint table in the queried database may be a parameter available to the querying entity so that the size of the query data fingerprint table can be determined, or the size of the fixed size data fingerprint table in the queried database may be determined by requesting or otherwise receiving information from the database.

[0051]

[0054] For example, referring to Figure 3, there is a schematic diagram of a method 300 for querying a database using a similarity search system. The system includes a data fingerprint table 310 in the database to be queried, the table having a fixed size that encompasses the fingerprints FingerPrintx to FingerPrinti of all data items. The size FingerPrintx to FingerPrinti of the data fingerprint table 310 is information known or available to the querying entity. Alternatively, the size FingerPrinti of the data fingerprint table 310 may be secure information such that the system uses it to generate the query data fingerprint table without revealing the secure information to the querying entity.

[0052]

[0055] A query data fingerprint table 320 is then generated such that Query-FingerPrint is repeated for each row, with the number of rows equal to the size FingerPrintx to FingerPrinti of the data fingerprint table 310. This allows for row-by-row similarity comparison of the query against all data in the database.

[0053]

[0056] Then, in step 170 of the method, the system compares the query against the data in the database. According to an embodiment, the system compares the homomorphically encrypted fixed length query data fingerprint against each of the homomorphically encrypted fixed length data fingerprints using a fixed size query data fingerprint table and a fixed size database table. The comparison results in a generated distance between the query and each data item.

[0054]

[0057] Referring again to FIG. 3, there is a schematic diagram of a method 300 for querying a database using a similarity search system in one embodiment. The system includes a data fingerprint table 310 of size FingerPrintx to FingerPrinti, and a query data fingerprint table 320 of the same size, populated with Query-FingerPrint repeated in each row. As each row of the data fingerprint table 310 is compared against each row of the query data fingerprint table 320, a distance Distance between the query and each data item is generated. The Distance is stored in a distance table 330 of size Distancex to Distancei, which is the same size as the data fingerprint table 310 and the query data fingerprint table 320. Thus, for example, Distancex is the distance between FingerPrintx and Query-FingerPrint, Distancei is the distance between FingerPrinti and Query-FingerPrint, and so on. According to an embodiment, the distance table 330 is also homomorphically encrypted.

[0055]

[0058] According to an embodiment, once both the dataset table and the query table are encoded as plaintext and homomorphically encrypted, the system can search for matches or otherwise determine distances by a compare, add, multiply algorithm, although other methods are possible. For example, given a ciphertext matrix D for i ranging from 1 to V, iFor a data set table encrypted as a vector of x, the system utilizes Equation 3 below:

number

[0056]

[0059] Similarly, according to an embodiment, each query table X is divided into V ciphertext matrices X for each query table i. i and the system performs the following calculations through row j of the matrix:

number

[0057]

[0060] According to an embodiment, this approach involves computing the squared Euclidean distance for each row of the matrix. According to an embodiment, row indices that result in zero or a small number are now ideal targets for duplicates or highly similar records.

[0058]

[0061] According to an embodiment, this approach may leak to a query inquirer performing operations on the homomorphically encrypted data some sensitive data, such as the resulting Euclidean distance information, which may be used by a potential malicious actor to infer other bit vector values ​​present in the homomorphically encrypted table. The smaller the Euclidean distance, the more likely this leakage is. However, as long as the bit vectors preserve locality, they are not explicitly reversible. An attacker may be able to extract knowledge about what types of records are in the homomorphically encrypted dataset, but will not be able to gain knowledge of any specific records.

[0059]

[0062] In step 180, the system uses the generated distance between the homomorphically encrypted fixed-length query data fingerprint and each of the homomorphically encrypted fixed-length data fingerprints to identify one or more data files in the database that have a minimum generated distance, no distance, or a distance greater than or less than a predetermined threshold. According to an embodiment, the system or query includes a request, parameter, or user setting that identifies a desired distance between the homomorphically encrypted fixed-length query data fingerprint and the homomorphically encrypted fixed-length data fingerprints in the database. For example, the desired distance may be 0 to identify the same or nearly the same data. As another example, the desired distance may be as small as possible to identify data that is as close to the query as possible.

[0060]

[0063] According to an embodiment, the system or query includes a request, parameter, or user setting that identifies a desired number of data items to be identified in the database using the query. For example, the system or query includes information or parameters that result in y items being returned in response to the query that are the closest y data items.

[0061]

[0064] Referring again to FIG. 3, there is a schematic diagram of a method 300 for querying a database using a similarity search system in one embodiment. The system includes a data fingerprint table 310 of size FingerPrintx to FingerPrinti and a query data fingerprint table 320 of the same size populated with Query-FingerPrint repeated in each row. As each row of the data fingerprint table 310 is compared against each row of the query data fingerprint table 320, a distance Distance between the query and each data item is generated. Then, in step 180 of the method, one or more items Distancex to Distancei in the distance table 330 are identified as being the most responsive distance to the query. For example, according to an embodiment, any Distance=0 is responsive to the query, and the FingerPrint that results in Distance=0 is identified as responsive to the query. As another example, any Distance less than a predetermined, programmatically, or user-defined threshold (or any Distance greater than the threshold) identifies one or more FingerPrints responsive to the query.

[0062]

[0065] In step 190 of the method, the system provides a response to the query or other report to the user via the user interface 240 of the similarity search system 200, including an identification of one or more data files that have been identified by the similarity search as being suitably similar to the query. According to an embodiment, the system displays the report on a display of the system. The display includes information about the query, the response data, and / or the one or more parameters utilized for the similarity search. Other information is possible. Alternatively, the report may be communicated to another device by wired and / or wireless communication. For example, the system communicates the report to a mobile phone, a computer, a laptop, a wearable device, and / or any other device configured to enable display and / or other communication of the report.

[0063]

[0066] According to an embodiment, the homomorphic encryption system is implemented in C++ using the SEAL library from Microsoft, although other implementations are possible. The implementation can utilize the Brakerski / Fan-Vercauteren scheme for homomorphic ciphertext (see Fan et al., "Somewhat Practical Fully Homomorphic Encryption," Cryptology ePrintArchive, Report 2012 / 144). The test creates compressed fingerprints for all files and calculates a similarity index between all files based on these fingerprints.

[0064]

[0067] Working Example

[0068] The following is an example of a similarity search using the systems and methods described or otherwise contemplated herein, it being understood that this is merely one possible application or embodiment of the systems and methods described or otherwise contemplated herein and is thus a non-limiting example.

[0065]

[0069] In this embodiment, the systems and methods described or otherwise envisioned herein are utilized to compare genomic queries against one or more reference genomes. Also, one important use case for secure updates, for example of databases, is to determine which cases should be re-evaluated without leaking actual case information. In many situations, whether knowledge databases or active patient cases, active cases rely on underlying references that are subject to change. For example, referring to FIG. 4, case 400 where a genomic analysis 410 is compared against various references 410 (Reference_a, Reference_b, . . . Reference_n). Indeed, in genomic knowledge databases and user cases, the focal case may consist of the definitive reference that drives the understanding of the genomic analysis.

[0066]

[0070] One clear pattern in genomics is that the underlying references are frequently modified. These references leak privacy-sensitive data such as diseases associated with the case, so assessing such modifications and their impact on genomic analysis is a valuable use case for homomorphic encryption. Therefore, to enable homomorphic encryption and the search methods and systems described or otherwise envisioned herein, we propose the following solution architecture. For example, a vector GA of fingerprints of references for genomic analysis.

number

[0071] and consider a database of up-to-date references, RD. RD= <r d1 ,...r dn > (Formula 6)

[0067]

[0072] For these two vectors GA and RD, we perform the Euclidean distance operation as described here. To quickly analyze whether any notable changes have occurred, the following simple calculation can be performed:

number

[0068]

[0073] If there is a significant difference, then there is a change and the information can be reported. If there is a change or difference, then one or more actions may be triggered, which may be in response to a reference or a query, as desired.

[0069]

[0074] Referring to Figure 2, a schematic diagram of a similarity search system 200, according to an embodiment. System 200 may be any system described or otherwise envisioned herein and may include any components described or otherwise envisioned herein. It will be understood that Figure 2 is in some respects an abstraction, and the actual organization of the components of system 200 may be different and more complex than that illustrated.

[0070]

[0075] According to an embodiment, system 200 includes a processor 220 capable of executing instructions stored in memory 230 or storage 260 or otherwise processing data, for example to perform one or more steps of the methods. Processor 220 is formed from one or more modules. Processor 220 may take any suitable form, including but not limited to a microprocessor, a microcontroller, multiple microcontrollers, circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a single processor, or multiple processors.

[0071]

[0076] The memory 230 may take any suitable form, including non-volatile memory and / or RAM. The memory 230 may include various memories, such as, for example, L1, L2, or L3 caches, or system memory. Thus, the memory 230 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read-only memory (ROM), or other similar memory devices. The memory may store, among other things, an operating system. The RAM is used by the processor for temporary storage of data. According to an embodiment, the operating system includes code that, when executed by the processor, controls the operation of one or more components of the system 200. It will be apparent that in embodiments in which the processor implements one or more of the functions described herein in hardware, software that is described as corresponding to such functionality in other embodiments may be omitted.

[0072]

[0077] User interface 240 includes one or more devices for enabling communication with a user. The user interface may be any device or system that allows information to be transmitted and / or received, including a display, a mouse, and / or a keyboard for receiving user commands. In some embodiments, user interface 240 includes a command line interface or a graphical user interface that is presented to a remote terminal via communication interface 250. The user interface may be collocated with one or more other components of the system, or may be located remotely from the system and communicate via a wired and / or wireless communication network.

[0073]

[0078] The communication interface 250 includes one or more devices for enabling communication with other hardware devices. For example, the communication interface 250 includes a network interface card (NIC) configured to communicate according to an Ethernet protocol. Additionally, the communication interface 250 implements a TCP / IP stack for communication according to the TCP / IP protocol. Various alternative or additional hardware or configurations for the communication interface 250 will be apparent.

[0074]

[0079] Storage 260 includes one or more machine-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, or similar storage media. In various embodiments, storage 260 stores instructions for execution by processor 220 or data on which processor 220 operates. For example, storage 260 stores operating system 261 for controlling various operations of system 200.

[0075]

[0080] It will be apparent that the various information described as being stored in storage 260 may additionally or alternatively be stored in memory 230. In this regard, memory 230 may be considered to constitute a storage device, and storage 260 may be considered to be a memory. Various other arrangements will be apparent. Furthermore, both memory 230 and storage 260 may be considered to be non-transitory machine-readable media. As used herein, the term non-transitory is understood to include all forms of storage, including both volatile and non-volatile memory, while excluding transitory signals.

[0076]

[0081] Although system 200 is shown as including one of each of the described components, various components may be multiple in various embodiments. For example, processor 220 may include multiple microprocessors configured to independently perform the methods described herein, or configured to perform steps or subroutines of the methods described herein, such that the multiple processors cooperate to achieve the functionality described herein. Furthermore, when one or more components of system 200 are implemented in a cloud computing system, various hardware components may belong to separate physical systems. For example, processor 220 may include a first processor in a first server and a second processor in a second server. Many other variations and configurations are possible.

[0077]

[0082] The system includes or is in communication with a query database 270. The query database is a database of any information, such as a medical database. The query database may be generated by the similarity search system or may be generated by another system. The database structure includes at least a fixed-size database table populated with a random sequence of homomorphically encrypted fixed-length data fingerprints, each of which represents a data file in the database. The fixed-size database table may be generated using known methods for generating data tables. The homomorphically encrypted fixed-length data fingerprints may be generated in accordance with methods described below and herein or otherwise envisioned.

[0078]

[0083] According to an embodiment, storage 260 of system 200 stores one or more algorithms, modules, and / or instructions for performing one or more functions or steps of methods described or otherwise envisioned herein. Among many possible instructions and / or data, for example, the system includes query shaping instructions 262, similarity search instructions 263, and / or reporting instructions 264, among other instructions or data.

[0079]

[0084] According to an embodiment, the query shaping instructions 262 instruct the system to receive and / or process an encrypted query to be utilized by the similarity search system. For example, the system may receive an encrypted query to be queried against a database, where the query may be any data or anything else that may be used in a search or otherwise queried against the database. The query may be received at the similarity search system, such as through a user interface. Alternatively, the query may be received from a remote system in wired and / or wireless communication with the similarity search system.

[0080]

[0085] According to an embodiment, the query shaping instructions 262 further instruct the system to generate a fixed length query data fingerprint from the received query. The fixed length query data fingerprint is generated using known methods for generating fingerprints from data. According to an embodiment, the fixed length data fingerprint includes a fixed length floating point valued feature vector.

[0081]

[0086] According to an embodiment, the query shaping instructions 262 further instruct the system to homomorphically encrypt the fixed-length query data fingerprint. The fixed-length query data fingerprint is homomorphically encrypted using any suitable method for homomorphic encryption, including but not limited to methods described or otherwise envisioned herein.

[0082]

[0087] According to an embodiment, the query shaping instructions 262 further instruct the system to generate a fixed-size query data fingerprint table, where each row is the same homomorphically encrypted query data fingerprint. Each row of the query data fingerprint table will be compared against each row of the randomly ordered homomorphically encrypted data fingerprints, where each data fingerprint represents a data file in the database. Thus, the size of the fixed-size query data fingerprint table is determined by the size of the fixed-size data fingerprint table in the database being queried. Once generated, the fixed-size query data fingerprint table may be immediately utilized or may be stored in local or remote storage for later querying.

[0083]

[0088] According to an embodiment, the similarity search instructions 263 instruct the system to compare the query against the data in the database. According to an embodiment, the system uses a fixed-size query data fingerprint table and a fixed-size database table to compare the homomorphically encrypted fixed-length query data fingerprint against each of the homomorphically encrypted fixed-length data fingerprints. The comparison results in a generated distance between the query and each data item.

[0084]

[0089] According to an embodiment, the similarity search instructions 263 also instruct the system to utilize this generated distance between the homomorphically encrypted fixed-length query data fingerprint and each of the homomorphically encrypted fixed-length data fingerprints to identify one or more data files in the database that have a minimum generated distance, no distance, or a distance greater than or less than a predetermined threshold. According to an embodiment, the system or query includes a request, parameter, or user setting that identifies a desired distance between the homomorphically encrypted fixed-length query data fingerprint and the homomorphically encrypted fixed-length data fingerprints in the database. For example, the desired distance may be 0 to identify data that is the same or nearly the same. As another example, the desired distance may be as small as possible to identify data that is as close to the query as possible.

[0085]

[0090] According to an embodiment, the reporting instructions 264 instruct the system to generate and provide to the user via the user interface a report including an identification of one or more data items identified by the similarity search as suitably similar to the query. According to an embodiment, the system displays the report on a display of the system. The display includes information regarding the identified data files, the query, and / or the one or more parameters utilized for the similarity search. Other information is possible. Alternatively, the report may be communicated to another device via wired and / or wireless communication. For example, the system communicates the report to a mobile phone, a computer, a laptop, a wearable device, and / or any other device configured to enable display and / or other communication of the report.

[0086]

[0091] According to an embodiment, the similarity search system is configured to process thousands or millions of data items in a database, including generating a fingerprint for each data item, homomorphically encrypting each data item, and creating a table including each homomorphically encrypted fixed-length data fingerprint. The similarity search system further generates a fingerprint for a query, homomorphically encrypts the query, and generates a table including the homomorphically encrypted fixed-length query data fingerprint. The similarity search system further compares the homomorphically encrypted fixed-length query fingerprint against each homomorphically encrypted fixed-length data fingerprint, thereby generating a distance for each comparison. This requires processing millions of data points. Thus, generating a distance between a query and each data item in a database involves a process having an amount of calculations and analysis that a human brain cannot complete in a lifetime, or even in multiple lifetimes. Furthermore, by providing faster and more secure similarity searches, the methods and systems described or otherwise envisioned herein represent a technical improvement over prior systems that cannot perform the same functionality at comparable speeds and in the same manner.

[0087]

[0092] All definitions, and as used herein, are to be understood as governing dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0088]

[0093] As used in the specification and claims, the singular articles "a," "an," and "the" are to be understood as meaning "at least one," unless expressly stated to the contrary.

[0089]

[0094] As used in the specification and claims, the term "and / or" shall be understood to mean "either or both" of the combined elements, i.e., the elements are present in combination in some cases and disjointed in other cases. Multiple elements listed with "and / or" shall be interpreted in the same manner, i.e., "one or more" of the combined elements. Other elements may optionally be present other than the elements expressly identified by the "and / or" clause, whether related or unrelated to those elements expressly identified.

[0090]

[0095] As used herein and in the claims, "or" shall be understood to have the same meaning as "and / or" defined above. For example, when separating items in a list, "or" or "and / or" shall be interpreted as inclusive, i.e., including at least one, but also including more than one of a plurality or list of elements, and optionally additional items not listed. Only terms expressly stated to the contrary, such as "only one of" or "exactly one of," or, when used in the claims, "consisting of," shall refer to the inclusion of exactly one element of a plurality or list of elements. In general, as used herein, the term "or" shall be interpreted as indicating exclusive alternatives (i.e., "one or the other, but not both") only when preceded by terms of exclusivity, such as "either," "one of," "only one of," or "exactly one of."

[0091]

[0096] As used herein and in the claims, the phrase "at least one" in reference to a list of one or more elements shall be understood to mean at least one element selected from any one or more elements in the list of elements, and does not necessarily include at least one of each element expressly listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows for the optional presence of elements other than those expressly identified in the list of elements to which the phrase "at least one" refers, whether related or unrelated to those elements expressly identified.

[0092]

[0097] Also, unless expressly stated to the contrary, in any claimed method including two or more steps or acts, it is to be understood that the order of the method steps or acts is not necessarily limited to the order in which the method steps or acts are described.

[0093]

[0098] In the claims and the foregoing specification, all transitional phrases such as "having," "including," "carrying," "having," "containing," "involving," "holding," "consisting of," and the like, are to be understood to be open-ended, i.e., meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively.

[0094]

[0099] Although several embodiments of the invention have been described and illustrated herein, those skilled in the art will readily conceive of various other means and / or structures for performing the functions and / or obtaining one or more of the results and / or advantages described herein, and each such variation and / or modification is deemed to be within the scope of the embodiments of the invention described herein. More generally, those skilled in the art will readily recognize that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the specific application or applications in which the teachings of the invention are used. Those skilled in the art will recognize, or be able to ascertain using only mechanical experimentation, many equivalents to the specific embodiments of the invention described herein. Thus, it is to be understood that the above embodiments are presented by way of example only, and that within the scope of the appended claims and their equivalents, embodiments of the invention may be practiced otherwise than as specifically described and claimed. Embodiments of the invention according to the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Furthermore, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure, unless such features, systems, articles, materials, kits, and / or methods are mutually inconsistent.

Claims

1. providing a database including a plurality of homomorphically encrypted fixed length data fingerprints, each of which represents a data file in the database, the plurality of homomorphically encrypted fixed length data fingerprints being randomly ordered in a fixed size database table; receiving an encrypted query to be queried against the database in a similarity search; generating a fixed length query data fingerprint from the received query; homomorphically encrypting the fixed length query data fingerprint; generating a fixed size query data fingerprint table, the size of the fixed size query data fingerprint table being the same as the size of the fixed size database table; comparing the homomorphically encrypted fixed length query data fingerprint against each of the homomorphically encrypted fixed length data fingerprints using the fixed size query data fingerprint table and the fixed size database table, each comparison producing a distance; using the generated distances between the homomorphically encrypted fixed length query data fingerprint and each of the homomorphically encrypted fixed length data fingerprints to identify one or more data files in the database having a minimum generated distance; reporting the identified one or more data files in the database; A method for encrypted similarity searching of a database, comprising:

2. The method of claim 1 , further comprising generating the plurality of homomorphically encrypted fixed-length data fingerprints.

3. 2. The method of claim 1, further comprising: generating the fixed size database table including the plurality of homomorphically encrypted fixed length data fingerprints, wherein the plurality of homomorphically encrypted fixed length data fingerprints are randomly ordered in the fixed size database table.

4. 2. The method of claim 1 , wherein the distance between each homomorphically encrypted fixed length data fingerprint and the homomorphically encrypted fixed length query data fingerprint is used to populate a distance table, the distance table being the same size as the fixed size query data fingerprint table and the fixed size database table.

5. The method of claim 1 , wherein the generated distances are homomorphically encrypted.

6. The method of claim 1 , wherein the reporting step further comprises the generated distance between the query and the identified one or more data files in the database.

7. 2. The method of claim 1, wherein the encrypted queries are updated versions of a database, and each fixed-length data fingerprint represents one version of the database or one version of a data file within the database.

8. 2. The method of claim 1, wherein the encrypted query is an updated version of a reference genome, and each fixed-length data fingerprint represents a reference genome.

9. a database including a plurality of homomorphically encrypted fixed length data fingerprints, each of the homomorphically encrypted fixed length data fingerprints representing a data file in the database, the plurality of homomorphically encrypted fixed length data fingerprints being randomly ordered in a fixed size database table; an encrypted query that is queried against the database in a similarity search; (i) generating a fixed length query data fingerprint from the query; (ii) homomorphically encrypting the fixed-length query data fingerprint; (iii) generating a fixed size query data fingerprint table, the size of the fixed size query data fingerprint table being the same as the size of the fixed size database table; (iv) comparing the homomorphically encrypted fixed length query data fingerprint against each of the homomorphically encrypted fixed length data fingerprints using the fixed size query data fingerprint table and the fixed size database table, each comparison producing a distance; (v) using the generated distances between the homomorphically encrypted fixed-length query data fingerprint and each of the homomorphically encrypted fixed-length data fingerprints to identify one or more data files in the database having a minimum generated distance; and (vi) generating a report that includes the one or more data files identified in the database. The processor and A system for encrypted similarity searching of a database comprising:

10. The system of claim 9 , further comprising a user interface that presents the generated report.

11. 10. The system of claim 9, wherein the processor is further configured to generate the plurality of homomorphically encrypted fixed length data fingerprints.

12. 10. The system of claim 9, wherein the processor is further configured to generate the fixed size database table including the plurality of homomorphically encrypted fixed length data fingerprints, the plurality of homomorphically encrypted fixed length data fingerprints being randomly ordered within the fixed size database table.

13. 10. The system of claim 9, wherein the distance between each homomorphically encrypted fixed length data fingerprint and the homomorphically encrypted fixed length query data fingerprint is used to populate a distance table, the distance table being the same size as the fixed size query data fingerprint table and the fixed size database table.

14. The system of claim 13 , wherein the distance table is homomorphically encrypted.

15. The system of claim 9 , wherein the minimum generated distance is determined by a pre-determined setting or is user-defined.

Images