Safely Match and Identify Patterns

Abstract

A framework is provided in which a querying institution, via a service provider, can request encrypted data (via a query entity) from a data-owning institution that stores the encrypted data. The framework uses homomorphic encryption. The data can be a gallery entity, and each element in the framework operates on the doubly encrypted information. The service provider compares the encrypted query entity representation from the querying institution with the encrypted gallery entity representation from the data-owning institution, resulting in a doubly encrypted value of an index between the corresponding compared representations. Based on the index, the querying institution obtains a result indicating whether the service provider is likely to have data similar to or the same as the query data in the query entity. The elements must communicate in order for the querying institution or the data-owning institution to obtain the plaintext information corresponding to the query entity.

Description

[Technical field]

[0001] The present invention relates generally to pattern matching and encryption, and more particularly to secure pattern matching and identification. [Background technology]

[0002] As businesses move data to the cloud to scale and process transactions efficiently, concerns and regulations related to data confidentiality and security are growing. In particular, placing unencrypted data in the cloud may allow others to access the data without authorization. On the other hand, encrypting the data brings its own challenges, since without the proper keys, the service provider hosting the information in the cloud has the ability to determine what is contained in the encrypted data. In other words, if a business wants to search for information within the encrypted data, the business would need to perform this search itself, unless the business gives the service provider the proper keys (thereby compromising security). Summary of the Invention

[0003] In an example embodiment, the method includes comparing an encrypted query entity representation with one or more encrypted gallery entity representations. The comparison of the encrypted query entity representation with one or more encrypted gallery entity representations results in a doubly encrypted value of an indicator between the corresponding compared representations. The encrypted query entity representation is homomorphically encrypted using a first private key, and the one or more encrypted gallery entity representations are homomorphically encrypted using a second private key, and the comparison is performed in the encrypted domain without decrypting the representations. The method also includes outputting a selected one or more of the one or more doubly encrypted values ​​of the indicator.

[0004] An additional example embodiment is a computer system including one or more processors and one or more memories including computer program code configured to cause the computer system to perform operations including comparing an encrypted query entity representation with one or more encrypted gallery entity representations, where the comparison of the encrypted query entity representation with the one or more encrypted gallery entity representations results in doubly encrypted values ​​of an indicator between corresponding compared representations, where the encrypted query entity representation is homomorphically encrypted using a first private key and the one or more encrypted gallery entity representations are homomorphically encrypted using a second private key, and the comparison is performed within the encrypted domain without decrypting the representations, and outputting a selected one or more of the doubly encrypted values ​​of the indicator.

[0005] A further embodiment is a computer program product comprising a computer readable storage medium having program instructions embodied thereon, the program instructions being executable by a computer system to cause the computer system to perform operations including comparing an encrypted query entity representation with one or more encrypted gallery entity representations, where the comparison of the encrypted query entity representation with the one or more encrypted gallery entity representations results in doubly encrypted values ​​of an indicator between corresponding compared representations, where the encrypted query entity representation is homomorphically encrypted using a first private key and the one or more encrypted gallery entity representations are homomorphically encrypted using a second private key, and the comparison is performed within the encrypted domain without decrypting the representations, and outputting a selected one or more of the one or more doubly encrypted values ​​of the indicator.

[0006] Another example embodiment is a method. The method includes sending, by a first computer system, a query including an encrypted representation of a query entity to a second computer system, where the encrypted representation of the query entity is homomorphically encrypted using a first homomorphic encryption scheme using a first private key. The method includes receiving, at the first computer system, from the second computer system, one or more doubly encrypted values ​​indicative of an index value used by the second computer system to compare the one or more encrypted representations of the gallery entities to the encrypted representation of the query entity, where the one or more doubly encrypted values ​​are encrypted using both the first homomorphic encryption scheme and a second homomorphic encryption scheme using a second private key. The method further includes decrypting, by the first computer system, the one or more doubly encrypted values ​​to produce corresponding one or more first partially decrypted values ​​that are decrypted with the first homomorphic encryption scheme. The method also includes receiving, at the first computer system, one or more second partially decrypted values ​​from the third computer system, where the one or more second partially decrypted values ​​correspond to distinct values ​​of the doubly encrypted value but decrypted with the second homomorphic encryption scheme. The method further includes performing, by the first computer system, a merge on the distinct values ​​of the corresponding first and second partially decrypted values ​​to form plain text values, where the plain text values ​​provide a value corresponding to an index, indicative of how well the corresponding representation of the encrypted gallery entity is deemed to match the representation of the encrypted query entity.

[0007] A further example embodiment is an exemplary apparatus including one or more processors and one or more memories containing computer program code, the one or more memories and the computer program code, together with the one or more processors, for causing the apparatus to: transmit, by a first computer system, a query including an encrypted representation of a query entity to a second computer system, where the encrypted representation of the query entity is homomorphically encrypted using a first homomorphic encryption scheme using a first private key; receive, by the first computer system, from the second computer system, one or more doubly encrypted values ​​indicative of a value of an index used by the second computer system to compare the one or more encrypted representations of the gallery entities to the encrypted representation of the query entity, where the one or more doubly encrypted values ​​are encrypted using both the first homomorphic encryption scheme and a second homomorphic encryption scheme using a second private key; The method is configured to cause the first computer system to perform operations including: decrypting, by one computer system, one or more first partially decrypted values ​​to produce corresponding one or more first partially decrypted values ​​that are decrypted with a first homomorphic encryption scheme; receiving, at the first computer system, one or more second partially decrypted values ​​from a third computer system, where the one or more second partially decrypted values ​​correspond to distinct values ​​of the doubly encrypted values ​​but are decrypted with the second homomorphic encryption scheme; and performing a merge, by the first computer system, on the distinct values ​​of the corresponding first and second partially decrypted values ​​to form a plaintext value, where the plaintext value provides a value corresponding to an index, indicative of how well a corresponding representation of the encrypted gallery entity is deemed to match a representation of the encrypted query entity.

[0008] A computer program product is an additional example embodiment, comprising a computer readable storage medium having program instructions embodied thereon, the program instructions being executable by a computer system to include a first computer system transmitting a query including an encrypted representation of a query entity to a second computer system, the encrypted representation of the query entity being homomorphically encrypted using a first homomorphic encryption scheme using a first private key, and receiving, at the first computer system, from the second computer system, one or more doubly encrypted values ​​indicative of an index value used by the second computer system to compare the one or more encrypted gallery entity representations to the encrypted representation of the query entity, the one or more doubly encrypted values ​​indicative of an index value used by the second computer system to compare the one or more encrypted gallery entity representations to the encrypted representation of the query entity, the one or more doubly encrypted values ​​being homomorphically encrypted using a second homomorphic encryption scheme using the first homomorphic encryption scheme and a second private key. receiving, by the first computer system, one or more doubly encrypted values ​​from the third computer system, the one or more second partially decrypted values ​​corresponding to distinct values ​​of the doubly encrypted values ​​but decrypted with the second homomorphic encryption scheme; and performing a merge by the first computer system on the distinct values ​​of the corresponding first and second partially decrypted values ​​to form a plaintext value, the plaintext value providing a value corresponding to an index indicating how well the corresponding representation of the encrypted gallery entity is deemed to match the encrypted representation of the query entity.

[0009] Another example embodiment is a method that includes receiving, at a first computer system, from a second computer system, one or more doubly encrypted values ​​indicative of an index value used by the second computer system to compare one or more encrypted gallery entity representations to an encrypted query entity representation. The one or more doubly encrypted values ​​are encrypted using both a first homomorphic encryption scheme using a first private key and a second homomorphic encryption scheme using a second private key. The method includes decrypting, by the first computer system, the one or more doubly encrypted values ​​to create corresponding one or more first partially decrypted values ​​that are decrypted with the second homomorphic encryption scheme. The method also includes outputting, to a third computer system, the one or more first partially decrypted values ​​that are decrypted with the second homomorphic encryption scheme.

[0010] Another exemplary apparatus includes one or more processors and one or more memories containing computer program code configured, in conjunction with the one or more processors, to cause the apparatus to perform operations including receiving, at a first computer system, from a second computer system one or more doubly encrypted values ​​indicative of an index value used by the second computer system to compare one or more encrypted gallery entity representations to encrypted query entity representations, where the one or more doubly encrypted values ​​are encrypted using both a first homomorphic encryption scheme using a first private key and a second homomorphic encryption scheme using a second private key, decrypting, by the first computer system, the one or more doubly encrypted values ​​to create corresponding one or more first partially decrypted values ​​that are decrypted with the second homomorphic encryption scheme, and outputting the one or more first partially decrypted values ​​that are decrypted with the second homomorphic encryption scheme to a third computer system.

[0011] An additional example embodiment is a computer program product comprising a computer readable storage medium having program instructions embodied thereon, the program instructions being executable by a computer system to cause the computer system to perform operations including receiving, at a first computer system, from a second computer system one or more doubly encrypted values ​​indicative of index values ​​used by the second computer system to compare one or more encrypted gallery entity representations with the encrypted query entity representations, where the one or more doubly encrypted values ​​are encrypted using both a first homomorphic encryption scheme using a first private key and a second homomorphic encryption scheme using a second private key; decrypting, by the first computer system, the one or more doubly encrypted values ​​to create corresponding one or more first partially decrypted values ​​that are decrypted with the second homomorphic encryption scheme; and outputting the one or more first partially decrypted values ​​decrypted with the second homomorphic encryption scheme to a third computer system. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates plain text matching and identification against a gallery of face images. [Diagram 2] FIG. 1 illustrates matching and identification within an encrypted domain against a gallery of face images. [Diagram 3] 1 is a block diagram of an exemplary system suitable for secure pattern matching and identification, in accordance with an example embodiment. [Figure 3A] FIG. 2 is a block diagram of another exemplary system suitable for secure pattern matching and identification, in accordance with an example embodiment. [Figure 4] 1 is a block diagram of a flowchart of an exemplary method for secure pattern matching and identification, in accordance with an example embodiment. [Diagram 5] FIG. 1 illustrates a matching algorithm, according to an example embodiment. [Figure 6A]FIG. 13 is a diagram of the algorithm used to find the index of a matching identity for a homomorphically encrypted number. [Figure 6B] FIG. 1 is a diagram of the algorithm used to find the square root of two homomorphically encrypted numbers. [Figure 6C] FIG. 1 is a diagram of the algorithm used to find the values ​​of min(a,b) and max(a,b). [Figure 7] FIG. 1 is a block diagram of one possible non-limiting example system in which example embodiments may be practiced. [Figure 8] FIG. 1 illustrates a cloud computing environment in accordance with an example embodiment. [Figure 9] FIG. 2 illustrates an abstract model layer according to an example embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following abbreviations which may appear in the specification and / or drawings are defined as follows:

[0014] FHE (fully homomorphic encryption)

[0015] HE (homomorphic encryption)

[0016] I(index) Index

[0017] I / F(interface)

[0018] MFHE (multi-key fully homomorphic)

[0019] SMC (secure multi-party computation)

[0020] SV(similarity value) similarity value

[0021] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. All of the embodiments described in this Detailed Description are example embodiments provided to enable any person skilled in the art to make or use the invention, but not to limit the scope of the invention, which is defined by the claims.

[0022] As previously mentioned, companies that are moving data to the cloud, for example to scale and process transactions efficiently, must address growing concerns and regulations related to data confidentiality and security. Encryption toolsets are one means of moving data to the cloud while providing tangible guarantees of security and confidentiality. However, as previously mentioned, placing encrypted data in the cloud or a similar storage location makes it more difficult to search. The techniques presented below, in example embodiments, provide an end-to-end encrypted solution to certain "matching" applications on the cloud or other suitable locations.

[0023] For example, what an enterprise might want to do is perform matching and identification on plain text (unencrypted information). Consider a useful class of applications called "identification" that includes the following:

[0024] (1) A pre-populated representation of one or more entities, collectively referred to as a gallery.

[0025] (2) Given a new entity representation (called a query or probe), the system needs to evaluate which of the entities in the gallery are most "similar" to the query entity.

[0026] (3) This evaluation involves whether the query entity is sufficiently similar to the gallery entity to determine that a "match" has been found.

[0027] (4) Representations can be structured (e.g., "name", ss#) or unstructured (e.g., image).

[0028] Example applications include facial recognition, license plate recognition, etc. FIG. 1 illustrates facial recognition, matching and identification in plain text against a gallery of facial images. In this example, a database 110 may be on the cloud, and a gallery 115 contains many facial images, of which images 1-18 are shown. The business ends a query 130 (also called a probe) with a matching request 120 (shown as "match") with the corresponding image 125. For example, there may be facial security enabled to access employee information, and the business needs to verify that this employee has access to employee information.

[0029] The evaluation performed uses the following equation:

[0030] s(q,g i )>T (1)

[0031] Where:

[0032] q: a representation of the query entity,

[0033] g i : a representation of the i-th gallery entity,

[0034] s: similarity function, and

[0035] T: threshold value.

[0036] In this example, the fifth image (image 5) matches input image 125 based on the similarity function s(·) according to equation (1) above, so the result is "yes" (i.e., there is a match), as indicated by reference numeral 140 and s(q,g5)>T (see reference numeral 145). Image 5 is output, as indicated by reference numeral 150, possibly along with other structured information (e.g., an access level to indicate whether the employee has access permissions to employee information).

[0037] While this benefits the enterprise, it also means that a lot of personal data (especially images of employees in this example) is beyond the enterprise's control. Indeed, with regard to the example of facial images, there have been recent confidentiality concerns regarding facial recognition. One concern involves reports indicating that certain government agencies have been found to be using databases for facial recognition searches. Specifically, the government organizations have been using photos from driver's licenses. However, if the enterprise were to place facial images as plaintext in the cloud or other locations where they could be accessed by anyone, the enterprise would be at risk for similar confidentiality issues.

[0038] Additionally, many regulations have been launched to address confidentiality and impose fines or take other actions for non-compliance. For example, the European Union's General Data Protection Regulation (GDPR) came into effect as of May 25, 2018, and imposes heavy fines of up to 4% of annual revenue on organizations for non-compliance. Many companies operate within regulated industries and handle sensitive data, adding significant costs to their business.

[0039] With these concerns in mind, it is possible to encrypt the data and perform searches on the encrypted data. Figure 2 illustrates matching and identification within an encrypted domain. Relative to Figure 1, the matching in Figure 2 is similar to the unencrypted matching, except that the gallery entities and the query entities are encrypted and the results are available in encrypted form that must be decrypted.

[0040] This is illustrated in Figure 2, where the plaintext (i.e., unencrypted) gallery 115 is encrypted (260) to become the encrypted gallery 215 in the database 110-1. The encryption function for the encryption operation 260 is E(g i ,K g ), where E(·) is the encryption function and g i is the representation of the i-th gallery entity, and K g is the key used by the encryption function to encrypt the images used in the gallery. The encrypted gallery 215 contains encrypted images 1-1 through 18-1, each of which is an encrypted version of the corresponding plaintext image 1 through 18.

[0041] The company uses the system to encrypt (255) the input image 125, creating an encrypted input image 125-1. q ), where E(·) is an encryption function, q is a representation of the query entity (e.g., including the encrypted input image 125-1), and K q is the key used for the query and used by the encryption function to encrypt the query q into the encrypted query 130-1.

[0042] The result is an encrypted image 270 that is returned to the business. The result is created using the following equation:

[0043] E(s(q,g i ),K g,K q )>E(T,K g ,K q ) (2)

[0044] Where:

[0045] q: a representation of the query entity,

[0046] g i : a representation of the i-th gallery entity,

[0047] s: similarity function,

[0048] T: threshold,

[0049] E(·): Encryption function

[0050] K g : the key used to encrypt the gallery, and

[0051] K q : The key used to encrypt the query.

[0052] In this example, we encrypt the fifth image (image 5) based on the similarity function s(·) according to equation (2) above. 5 Since -1 matches the encrypted input image 125-1, the result is E(s(q,g5),K g ,K q ) The company has a key K q 2 to decode 265 the result 270 using the quantized image 5 to obtain image 5 as indicated by reference numeral 150.

[0053] There are two competing interests here: on the one hand, the enterprise needs to control the access rights to the data, and especially to private data, such as by using encryption in Figure 2, and on the other hand, some government or other agency may have legitimate reasons to request such data. So one problem is that the enterprise (the agency that owns the data) needs to control the data while still allowing agencies (the agency that is making the inquiry) to make inquiries for legitimate reasons, such as to perform authorizations, to determine compliance (with regulations), or for other legitimate reasons.

[0054] Example embodiments herein address these issues by providing a framework whereby a querying authority, which has a legitimate reason to request the data, can request encrypted data, e.g., via a service provider, from a data owning authority that stores the encrypted data. FIG. 3 illustrates an exemplary system suitable for secure matching and identification of patterns, according to example embodiments. The framework allows a querying authority 310 to request a match from a service provider 320 on query data 305. The querying authority 310 obtains a result (similarity value 390) indicating whether the service provider 320 is likely to have data similar to or the same as the query data 305. Additional verification 370 may be performed with the approval of the data owning authority 330. This is described in more detail below.

[0055] As an example, as previously described, a querying agency is permitted in this framework 300 to make queries for legitimate reasons, such as to perform authorization, to determine compliance (e.g., with regulations), or for other legitimate reasons. As an example, the querying agency 310 may be the Federal Bureau of Investigation (FBI), while the agency owning the data may be another part of the federal government. The FBI may need to know if a photo of a person taken by the FBI is a wanted person. The FBI may use this limited query to determine if the person in the photo is indeed a wanted person. Note that the verification of block 370 may pass (e.g., the facial images are similar) or fail (e.g., the facial images are not similar, such as being facial images of different people).

[0056] Note that the service provider 320 may allow queries from certain organizations, or only allow so many queries over a period of time, or impose other restrictions on the inquiring institution. The institution 330 owning the data may provide approval depending on the circumstances, or a court may provide approval for these queries. Restrictions and approvals are outside the scope of this disclosure. Instead, this disclosure provides a framework 300 for such (usually limited) queries occurring against encrypted data.

[0057] In additional detail, the framework includes three entities: a querying authority 310, a service provider 320 that may reside on a cloud, and a data owning authority 330. The querying authority 310 includes an encryption process 308, a decryption process 330, and a merging process 350, and may include a verification process 370. The service provider 320 includes a matching process 315. The data owning authority 330 includes an encryption process 320, a decryption process 330, and a merging process 350, and may include a verification process 370. 1, a decryption process 340, and a merging process 360. A data owning authority 330 owns the deployed data along with a service provider 320. A querying authority 310 would like to determine whether the query data 305 is contained within a (e.g., encrypted) database 389.

[0058] The operations performed by the entities of Figure 3 are best explained by example, which also includes Figure 4, which is a flow chart block diagram of an exemplary method for secure pattern matching and identification, according to an example embodiment. The following description refers to Figures 3 and 4, with reference numbers 3XX (where "X" is any integer) on Figure 3 (or Figure 3A) and reference numbers 4XX on Figure 4.

[0059] Note that the present techniques, such as those shown in Figures 3 and 4, leverage the techniques of multi-key fully homomorphic encryption (MFHE) to enable pattern matching within the encrypted domain. For more information on these multi-key techniques, see, among other places, Adriana Lopez-Alt, Eran Tromer, and Vinod Vaikuntanathan, “On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption”, Proceedings of the Forty-Fourth Annual ACM Symposium on Theory of Computing, ACM, 2012.

[0060] At block 410, the entity 330 that owns the data performs the encryption process 32. 1 By this, the private key K gThe data (e.g., gallery) entity (e.g., gallery data 322) is encrypted using a fully homomorphic technique using 326. It should be noted that anything that can be performed in fully homomorphic mode can also be performed in somewhat homomorphic mode, but the reverse may not be true. The data-owning entity 330 sends the resulting encrypted gallery data 316 to the service provider 320. The data-owning entity 330 also sends the public key 302 to the service provider 320. The fully homomorphic technique uses a private key K g 326 and the public key 302. Note that while a gallery of face images is used as the primary example herein, these techniques are not limited to this example. Furthermore, the term "gallery" may encompass a collection of objects or other entities in a database.

[0061] It is further noted that the representation may be the encrypted data of the images in the gallery (in the case of a gallery of images). However, the representation is more likely to be the encrypted data of the feature vectors of the face information, which may then allow for a better comparison. The representation may be received as such a feature vector (as the encrypted gallery data 316), but it may be possible to receive the encrypted gallery data 316 as images and convert these into feature vectors using homomorphic operations. That is, the gallery data 316 is encrypted. Given the encrypted images, it is possible to compute the encrypted feature vectors in the homomorphic domain, such that feature vectors can be created from images or text input into the homomorphism. The feature vectors can then be stored in the encrypted gallery 317 together with the corresponding images, for example as metadata for the corresponding images. Similarly, the encrypted query data 312 can also be encrypted images or encrypted feature vectors of images.

[0062] The service provider 320 places the encrypted gallery data 316 into an encrypted gallery 317 in a database 389. With reference to Figure 2, the encrypted gallery 317 can be thought of as the encrypted gallery 215 and the database 389 can be thought of as the database 110-1. Because encrypted data is essentially a series of binary data that forms a pattern, the encrypted gallery data 316 in the encrypted gallery 317 can be thought of as a pattern where it can be compared to other patterns, as described below.

[0063] At block 420 of FIG. 4, the querying authority 310 generates a key K q 325 to encrypt a representation of the query entity (e.g., query data 305). As shown in FIG. 5, multiple query entities may be present in a query. The querying authority 310 sends the resulting encrypted query data 312 to the service provider 320 as part of the query (also called a probe) 313. The querying authority 310 also sends the public key 301 to the service provider 320. The fully homomorphic technique uses a private key K q 325 and the public key 301.

[0064] In response to receiving the encrypted query data 312, the service provider 320 performs a matching process 315. In FIG. 4, the matching process 315 includes operations at blocks 430 and 440. At block 430, the service provider performs a comparison using a metric (e.g., distance, cosine distance, similarity metric) and a corresponding algorithm to find a value of the metric between the encrypted query / probe 313 and each of the encrypted representations of the entities in the gallery, and the resulting value is K g and K. q Exemplary algorithms that may be used are shown in Figures 6A, 6B, and 6C. The service provider 320 uses a private key K g and K.q Note that in addition to performing the distance calculation, it is possible to compare the calculated distance to a threshold without having access to K. More specifically, a service provider (SP) 320 may have access to a key K g 316, which contains gallery data encrypted according to q The query data 312 includes the query data encrypted according to . If two inputs to a computation such as a distance computation are encrypted using different keys, the output distance is encrypted by both keys. One option to achieve this property is to use the method of multi-key homomorphic encryption proposed in Hao Chen et al., “Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference”, The 26th ACM Conference on Computer and Communications Security (CCS 2019), DOI: 10.1145 / 3319535.3363207. More specifically, as explained in section 4 of the above reference, any operation between ciphertexts encrypted using different keys produces an extended ciphertext at the SP (service provider). The public keys 301 and 302 of each party can be combined with the corresponding evaluation key (a special encryption of the private key by each party alone) to generate a relinearization key for each party (the querying authority and the authority owning the data). By applying a complex process called relinearization using each relinearization key, the expanded ciphertext sent by the SP can be transformed by both parties into more conventional ciphertexts. These conventional ciphertexts can be partially decrypted and finally merged by both parties to obtain the final decrypted computation result.

[0065] Cosine distance is one exemplary distance metric that may be used. Note that the metric used effectively "ignores" that the data is encrypted. In other words, the metric does not need to be modified to be used for encrypted data. Note also that other "distance" metrics may be used. Many such metrics exist, such as the Hamming distance metric. However, alternatively, or possibly additionally, various "similarity" metrics and corresponding algorithms may be used. These are similar, but use different measures. For example, for distance metrics, the smallest distance between multiple items indicates the "best" match between the items, while for similarity metrics, the highest similarity metric indicates the best match. Many other distance measures (e.g., Hamming, Euclidean, Manhattan, Mahalanobis, etc.) or similarity measures (e.g., correlation) may be used.

[0066] At block 440, the service provider 320 determines which selected values ​​to modify and send to the querying institution 310 and the data owning institution 330. Blocks 430 and 440 are illustrated in the example embodiment of FIG. 5, described below.

[0067] Before proceeding with further description of FIG. 4, it is useful to describe an exemplary process that may be performed by the service provider 320. First, in FIG. qg , S.V. qg ), and the subscript qg denotes the private key K q 325 and K gFIG. 3A illustrates a double encryption according to a homomorphic encryption scheme corresponding to 326 (e.g., or their corresponding public keys 301, 302). However, the service provider 320 may instead generate multiple results 397, for reasons explained below. This is also illustrated by FIG. 3A, which is a block diagram similar to FIG. 3 of another exemplary system suitable for secure pattern matching and identification, according to an example embodiment. One such multiple results 397-1 of FIG. 3 is a vector of N indices and corresponding N similarity values ​​[(I1,SV1),(I2,SV2),...,(I N , S.V. N Another example is a vector of N indices and a corresponding vector of N similarity values ​​[I1,I2,...,I N ] and [SV1,SV2,...,SV N ]. The index is information that allows a corresponding similarity value to be uniquely accessed from other similarity values, such as in a database 389. The similarity value may correspond to an image (or may correspond to a feature vector, where the encrypted gallery data 316 is stored). Furthermore, the decryption in block 330 generates a plurality of indexes I g1...N and N similarity values ​​SV g1...N The notation I may include decrypting g1...N is used to save space, and (I g1 ,I g2 ,...,I gN ) is equivalent to the notation SV g1...N is used to save space, and (SV g1 , S.V. g2 ,...,SV gN ) Similar notation is used elsewhere in FIG. 3A. For example, decryption 340 may be performed using N indices I g1...N and N similarity values ​​SV q1...N The method may include decoding the

[0068] Next, with regard to this process, it is important to understand that the service provider 320 operates entirely within the encrypted domain, so while the service provider 320 may perform multiple calculations, implement algorithms, and use thresholds (as described below), the service provider never knows the results of these calculations.

[0069] Consider some simple examples. Suppose the service provider 320 computes a vector of similarity values ​​[SV1, SV2, SV3, SV4, SV5, SV6] after matching. Suppose the decision rule in step 440 (see block 311) transmits the top two similarity values ​​that are greater than a (doubly encrypted) threshold T. The service provider then computes max(SV i -T,0) (where max(A,B) selects the maximum value among A and B), sort the results, and hide entries other than the first two. The decision rule 311 may use a restriction, referred to herein as L. In Examples 1 and 2 below, L=2. The decision rule 311 and its restriction L may be set by the data owning authority 330, the querying authority 310, or the service provider 320, or a combination thereof. The restriction L may be a plaintext or a doubly encrypted value, depending on the implementation. EXAMPLES

[0070] Example 1 Assume SV2, SV4, and SV5 are greater than T, and SV2>SV5>SV4. In this case, the encrypted similarity vector can be [SV2,SV5,0,0,0,0], and the encrypted index vector would be [2,5,0,0,0,0]. These two results are sent as the result 397 to the querying institution and the institution owning the data, respectively. Note that zero ("0") is actually a doubly encrypted representation of zero. Also note that based on the index being used, the values ​​being used will change (e.g., the distance value may be "1" instead of zero).

[0071] Example 2 Assume that no similarity value is greater than T. In this case, the encrypted similarity vector will be [0,0,0,0,0,0] and the encrypted index vector will be [0,0,0,0,0,0,0]. These two results (with the first two zeros) are sent as the result397 to the querying institution and the institution owning the data, respectively.

[0072] Note that in both embodiments, the service provider has no knowledge of the contents of the result vector: only the querying authority and the authority that owns the data can jointly decrypt the result 397.

[0073] In a first embodiment, the querying authority 310 knows the top two similarity values ​​and the data owning authority 330 knows the indexes of the top two most similar matches. In a second embodiment, both authorities know that there were no matches that were similar enough.

[0074] In accordance with these, in block 450, which is an example of block 440, the service provider 320 uses the doubly encrypted threshold T to determine which encrypted query entities are sent as is and which are altered (e.g., set to zero), and outputs the encrypted results 397 to the querying authority 310 and the data owning authority 330. Note that the "doubly encrypted threshold" is a concept. In some implementations, the threshold can be explicitly encrypted using both public keys and used as described herein. In other implementations, a plaintext threshold can be used, but the result of the comparison is still doubly encrypted. This is due to the properties of the FHE scheme.

[0075] In this example, the encrypted result 397 also includes an index 318 of the corresponding matched gallery entry and a corresponding similarity value 319 (e.g., in database 389). The similarity value is an indication of "how similar" the matched gallery entry is to the encrypted query data 312. For example, one common scale used is from 0 (not similar) to 1 (very similar), with the similarity value falling within this range. Other ranges may be used. Note that Hao Chen et al.'s reference ("Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference") also describes an exemplary predictive model that may be used as a suitable algorithm for "similarity" determination.

[0076] However, it is important to note that these are not "raw" similarity values, but instead are doubly encrypted similarity values ​​319. The querying authority 310 cannot decrypt these similarity values ​​without assistance from the data owning authority 330 and also the querying authority 310. Furthermore, it is important to reiterate that the service provider 320 does not know if any gallery entity is sufficiently similar to the query, since the service provider 320 only sees the doubly encrypted comparison result 397. Only the querying authority 310 (with assistance from the data owning authority 330 and the querying authority) can see the result of the comparison in plain text form. Therefore, the decision to discard or keep the selected gallery entity is not made by the service provider 320. However, the service provider 320 limits the number of results sent using an algorithm and possibly a doubly encrypted threshold T (or limit, L, denoted as the number of results to send, which may be used with the decision rule 311).

[0077] Thus, all results 397 may be transmitted, but a particular subset of the encrypted results 397 may be transmitted. For example, assume that there are 1 million images (or feature vectors of images) in the encrypted gallery 317. This may result in an encrypted result 397 containing 1 million entries, each entry including an index and a corresponding similarity value. Instead of transmitting the entire 1 million entries, a doubly encrypted threshold T may be used to select which subset is transmitted, possibly together with a limit, L (as in Examples 1 and 2 above where the limit L indicates the number to be transmitted). This is indicated by block 455, where the similarity values ​​may be sorted based on the metric being used. For example, if the metric is a similarity metric, then the higher the number, the better. On the other hand, if the metric is a distance metric, then the lower the number, the better. The doubly encrypted threshold T may be used as a cutoff point to determine which values ​​are to be transmitted. For example, in the case of a similarity metric, similarity values ​​above a doubly encrypted threshold T may be transmitted, and values ​​below the doubly encrypted threshold T would not be transmitted from the service provider 320 to the querying authority 310, nor would they be transmitted to, for example, the data owning authority 330. Another embodiment would use a limit L, such that the doubly encrypted threshold T is used to determine which results are altered or not altered, but the limit L and decision rule 311 are used to determine which results are transmitted and which are not. For example, perhaps only the top 10 results (subject to the limit L) are transmitted, and a doubly encrypted threshold might be applied to all results to determine which results are altered (e.g., to zero) and which results are not altered.

[0078] With respect to blocks 430 and 440, one can imagine parameterizing the inputs and results, including the size of the output list, the thresholds, the distance (or other metric) functions used. This would allow further control over the behavior of the service provider 320. However, again, note that the service provider 320 only has the information in encrypted form (thresholds, result lists, queries, etc.). This information can be decrypted by either the querying authority 310 or the data owning authority 330 only (in the case of information encrypted by only one of these), or by both the querying authority 310 and the data owning authority 330 (in the case of information that is double encrypted or partially decrypted from double encryption). The important part to note is that none of the partial results can be inferred by the service provider 320. Instead, the service provider 320 only applies the encrypted similarities, encrypted thresholds, etc., but has no idea as to how many results qualify based on the specified thresholds.

[0079] In block 460, the double encrypted result 397 is decrypted by the data owning authority 330 and the querying authority 310. The data owning authority 330 uses the private key K g The doubly encrypted result 397 is decrypted using a decryption process 340 using 326. This results in the partially decrypted index I q and the partially decoded similarity value SV q These generate a result 341 containing the private key K g 326, but the private key K q For 325 it remains encrypted, which is the reason for the subscript "q".

[0080] The querying authority 310 obtains the private key K qThe doubly encrypted result 397 is decrypted using a decryption process 330 using 325. This results in the partially decrypted index I g and the partially decoded similarity value SV g These generate a result 331 containing the private key K q 325, but the private key K g It remains encrypted for 326, which is the reason for the subscript "g".

[0081] At block 470, the partially decoded similarity value SV q 385 is sent from the data-owning authority 330 to the querying authority 310 to receive the partially decrypted matched index I g 380 is sent from the querying institution 310 to the institution 330 owning the data.

[0082] In block 480, the independently encrypted results 380, 385 are decrypted by the querying institution and the institution owning the data. Specifically, if the index 375 is in plaintext, the partially decrypted matched index I is decrypted to create the index 375 of matched gallery entities. g 380 is decrypted by the data-owning authority 330. Similarly, if the similarity values ​​are in the clear, the partially decrypted similarity values ​​SV q 385 is decrypted by the querying authority 310 .

[0083] As a further example, the multiple values ​​passed so far by the client (in this case the querying authority 310) reflect the client's choice of threshold (e.g., defined by the limit L and the decision rule 311) and candidate list size of the response list in addition to the image. The candidate list size subject to the limit L defines the size of the encrypted results 397-1, 397-2, for example. These may also be present in applications such as search. The threshold and candidate list size are used by the service provider 320 to limit the number of responses, and they are meaningful to the querying authority 310. For example, the querying authority 310 may be interested in responses with more than 80% matches, including only the top 5 results. Another querying authority may be interested in a threshold of more than 0.5 and a candidate list size of 100. These are applied after the full results are obtained for the encrypted match.

[0084] In the multi-key FHE scheme used herein, the doubly encrypted ciphertexts cannot be decrypted using two successive decryption operations involving two keys 325, 326. Instead, the way to decrypt doubly encrypted ciphertexts with multi-key FHE is to partially decrypt them using the two keys separately and then merge the two partially decrypted results to obtain the plaintext. Therefore, the merging steps 350, 385 are required. See, for example, section 4.3 of “Distributed Decryption”, of Hao Chen, et al., “Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference”, Cryptology ePrint Archive: Report 2019 / 524. Section 4.3 of this reference describes an example of distributed decryption using two algorithms: partial decryption and merging. Similar or the same algorithms may be used for the decryption 330, 340 and merging 350, 360 steps herein.

[0085] At this point, the querying authority 310 has determined that there is a match with the query data 305 in the database 389 and can also determine a similarity value 390 for this match. Note that the similarity value 390 does not have to indicate a match. For example, if a similarity index is used, 0 can indicate no similarity and 1 can indicate similarity. Thus, a value of 0.15 can indicate that there is no match, while a value of 0.95 can indicate a match. Note that while the querying authority 310 only has the similarity value 390, it does not have the index 375 and therefore does not have access to the actual gallery data 316 corresponding to the index. Furthermore, even if the querying authority 310 has the index 375 and can access an image corresponding to that index from the encrypted gallery 317, the image is encrypted (as encrypted gallery data 316). Thus, the querying authority 310 only gets limited information about the face image.

[0086] As a result, as an optional operation, the querying agency 310 may request the matching gallery entities in plain text form from the data owning agency 330 for verification. See block 490. Once the querying agency determines that a match exists, the querying agency 310 will likely subsequently request that the data owning agency 330 (or other organization) provide the plain text data. For example, if the Federal Bureau of Investigation (FBI) finds a match with another federal agency's database, the FBI may initiate a court or other process to obtain the plain text data (of only the matched entities) from that agency. It is therefore important for the data owning agency 330 to know the index 375 of the match so that the data owning agency 330 can retrieve the correct data when a follow-up request comes from the querying agency 310. This process is illustrated in FIG. 3, where the matching gallery entities 395 are sent from the data owning agency 330 to the querying agency 310 in plain text. The querying institution 310 performs a verification process 370 to verify that the matched gallery entities 395 correspond to the query data 305. This step requires that the institution 330 that owns the data perform this action, and the approval schemes for this are outside the scope of this disclosure.

[0087] Further verification in block 370 involves the use of a partially decrypted matched index I375 by the querying authority 310 to generate a request for a plaintext index I375 (or multiple plaintext indexes I375) and the (plaintext) gallery data 395 corresponding to that plaintext index. g (or index) and index(I g , I q ) (or corresponding sets of indexes) (see block 371 in FIG. 3). This additional validation may involve the querying authority 310 sending a request for merging the "matching" similarity values ​​SV x (or similarity value SV xNote that the matching process may include selecting a vector of the similarity value of [SV2,SV5] and a corresponding index Ix (or set of indexes Ix). The matching similarity value or values ​​are based on the values ​​of the indices used, such as a similarity index or a distance index. In the case of Example 1 above, the querying authority obtained (as result 397-2) a vector of similarity value of [SV2,SV5] and index of [2,5]. The querying authority 310 could select only the similarity value of SV2 and index of 2 to send to the data owning authority 330. If permitted and desirable for the querying authority 310, this authority may send both the similarity value of [SV2,SV5] and index of [2,5] to the data owning authority 330. The data owning authority 330 would respond with plaintext gallery data 322 corresponding to either index 2 or indexes 2 and 5, as appropriate.

[0088] FIG. 3A illustrates a similar concept, where a querying authority 310 receives a single index ( g1 ) to the data-owning institution 330, which may then send a corresponding plaintext index 375 and corresponding gallery data 395. However, in another example, the querying institution 310 may send a request for multiple indexes (I g1...M )(I g1...M (I g1 ,...,I gM ) and M>1) to the data owning institution 330, which can send M corresponding plaintext indexes 375 and M corresponding gallery data 395 to the querying institution 310.

[0089] Note that the encrypted gallery data 316 is considered to be images containing facial information in this example. Although images are assumed herein, it is possible to apply the framework 300 to other data, such as employee records, as long as the records contain defined features that can be compared. Indeed, as explained immediately below, facial information is typically compared using feature vectors, which are mathematical representations of facial features in images. Thus, the comparison may be between vectors (or between matrices), or anything that can be represented as a vector (or matrix) may be used. That is, the technique is general and not limited to facial images. Facial images are merely an example.

[0090] Referring to FIG. 5, this diagram illustrates a matching algorithm in accordance with an example embodiment. This diagram is an example of the operations in blocks 430 and 440 of FIG. 4, for example. This diagram illustrates how the numbers are compressed so that the calculations can be performed efficiently. There are many other ways to compress these numbers. Similarly, the query can be compressed in various ways, but still the distances can be obtained as a single vector. Thus, FIG. 5 is only an example and not a limitation. In the example of FIG. 5, the gallery portion 317-1 includes four image feature vectors 520-1 to 520-4. That is, each column is a feature vector and corresponds to face information from one image and also corresponds to an index of the gallery. In this example, the probe 313 shows multiple images as encrypted query data 312, and the image feature vectors 530-1 to 530-4 correspond to face information from four different images.

[0091] A distance vector 510 is created for each pair of feature vectors 520, 530. That is, there will be 16 distance vectors 510. This illustrates the operation of block 430. Note that this operation can be performed by vector multiplication or, if supported, matrix multiplication.

[0092] Two distance vectors a 510-1 and b 510-2 are shown and the determination of which of the two vectors has the maximum value is made as follows (see also reference numeral 520): Max(a,b)=0.5*((a+b)+sqrt((ab)^2), where

number

[0093] A number of algorithms can be used to find the index of the matched identity, for example using the equation at reference number 520. One such algorithm is algorithm 6:MaxIdx proposed in reference Cheon, Jung Hee, et al., “Numerical Methods for Comparison on Homomorphically Encrypted Numbers”, IACR Cryptology ePrint Archive 2019 (2019): 417. FIG. 6A is a diagram of an algorithm used to find the index of the matched identity, according to an example embodiment. FIGS. 6B and 6C are additional diagrams of Cheon's algorithm that may be used. FIG. 6B is a diagram of an algorithm used to find the square root of two homomorphically encrypted numbers. FIG. 6C is a diagram of an algorithm used to find the values ​​of min(a,b) and max(a,b), where min(·) finds the minimum value and max(·) finds the maximum value. No changes need to be made to these algorithms, except that the inputs are doubly encrypted.

[0094] Referring to Figure 7, this figure illustrates a block diagram of one possible non-limiting exemplary system 700 in which example embodiments may be practiced. In Figure 7, a computer system 710 communicates wired and / or wirelessly with a wired and / or wireless network 797, which communicates with other computer systems 790 via the network 797. The computer system 710 may be accessible via one or more wired or wireless networks 797, may be implemented in a cloud 795, etc. The computer system 710 may alternatively or additionally be a server, although a client / server relationship is not required.

[0095] The computer system 710 includes one or more processors 720, one or more memories 725, one or more transceivers 730, one or more network interfaces (I / F) 745, and user interface circuitry 765, interconnected via one or more buses 727. Each of the one or more transceivers 730 includes a receiver Rx 732 and a transmitter Tx 733. The one or more buses 727 may be an address bus, a data bus, or a control bus, or a combination thereof, and may include any interconnect mechanism, such as a series of wires on a motherboard or integrated circuit, optical fiber, or other optical communication devices. The one or more transceivers 730 are connected to one or more antennas 728 (if a wireless system is used). The one or more memories 725 include computer program code 723.

[0096] The computer system 710 includes a control module 740 that includes one of components 704-1 and / or 740-2. The control module 740 performs the operations described herein, such as to perform any of the operations of the referring institution 310 or the service provider 320 described herein. The control module 740 may be implemented in a number of ways. The control module 740 may be implemented in hardware as a control module 740-1, such as implemented as part of one or more processors 720. The control module 740-1 may be implemented as an integrated circuit or by other hardware, such as a programmable gate array. In another example, the control module 740 may be implemented as a control module 740-2, implemented as computer program code 723 and executed by one or more processors 720. For example, the one or more memories 725 and the computer program code 723, along with the one or more processors 720, may be configured to cause the computer system 710 to perform one or more of the operations as described herein in response to obtaining and executing the computer program code 723. It should also be noted that the devices depicted in the computer system 710 are not limiting and that additional, different, or fewer devices may be used.

[0097] The user interface circuitry 765 communicates with one or more user interface elements 705, which may be integrally formed with the computer system 710 or may be external to but coupled to the computer system 710. The interface elements 705 include one or more of one or more cameras, one or more audio devices (microphones, speakers, etc.), one or more sensors (GPS sensors, fingerprint sensors, orientation sensors, etc.), one or more displays, or one or more keyboards, or a combination thereof. A display 711 is shown, which may be external or internal to the computer system 710. This list is not exhaustive or limiting, and other, different, or fewer elements may be used. A user 701-1 (a human being in this example) may interact with the computer system 710, for example, to cause the system 710 to perform a particular action. These actions may be triggered by computer system 710 in combination with or without actions by user 701-1. Computer system 710 communicates with other computer systems 790 via one or more wired or wireless networks 797 via one or both of wired link 777 and wireless link 778. Additionally, if computer system 710 is implemented within cloud 795, user 701-2 may use computer system 790 to interact with computer system 710 via wired and / or wireless networks 797. In this example, computer system 790 would include appropriate user I / F circuitry and user I / F elements (not shown) that would be similar to user I / F elements 705.

[0098] One example is when the computer system 710 is in the cloud 795 and thus is a cloud service provider 320. In that case, the other computer system 790 would be the querying institution 310 and the data owning institution 330. The control module 740 would perform the matching process 315 and other operations previously described as being performed by the service provider 320. In another example, the computer system 710 would be one of the querying institutions 310 or service providers 320 and the other computer system 790 would be the service provider 320, for example as a server connected to the computer system 710 via a network 797. .

[0099] When the service provider 320 is implemented in a cloud 395, Figures 8 and 9 provide additional information regarding cloud implementations. Figure 8 illustrates a cloud computing environment according to an example embodiment, and Figure 9 illustrates abstract model layers according to an example embodiment.

[0100] Referring now to FIG. 8, an exemplary cloud computing environment 50 is shown. Any or all of the cloud computing environment 50 implementing any of the embodiments herein may be considered to be a computer system 110. In other words, the computer system 110 may be considered to be a part of the cloud computing environment 50 implementing any of the embodiments herein. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers (e.g., a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof) may communicate. The nodes 10 may communicate with each other. The nodes 10 may be physically or virtually grouped together in one or more networks (not shown), such as a private cloud, a community cloud, a public cloud, or a hybrid cloud, or combinations thereof, as previously described herein. This allows the cloud computing environment 50 to provide an infrastructure, platform, and / or SaaS that does not require cloud consumers to maintain resources on local computing devices. It is understood that the types of computing devices 54A-N illustrated in Figure 8 are intended to be exemplary only, and that the computing nodes 10 and cloud computing environment 50 may communicate with any type of computer-controlled device via any type of network and / or network-addressable connection (e.g., a connection using a web browser).

[0101] Referring now to Figure 9, a set of functional abstraction layers provided by cloud computing environment 50 (Figure 8) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 9 are intended to be illustrative only and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0102] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframes 61, Reduced Instruction Set Computer (RISC) architecture based servers 62, servers 63, blade servers 64, storage devices 65, and networks and network components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0103] The virtualization layer 70 comprises an abstraction layer that can provide instances of virtual entities such as virtual servers 71 , virtual storage 72 , virtual networks including virtual private networks 73 , virtual applications and operating systems 74 , and virtual clients 75 .

[0104] As an example, the management layer 80 may provide the following functions: Resource provisioning 81 provides dynamic procurement of computational and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking and billing or invoicing for the utilization of resources within the cloud computing environment. As an example, the resources may include application software licenses. Security provides identity verification of cloud users and tasks, as well as protection of data and other resources. User portal 83 provides access to the cloud computing environment to users and system administrators. Service level management 84 provides allocation and management of cloud computational resources to meet required service levels. Service level agreement (SLA) planning and execution 85 provides pre-provisioning and procurement of cloud computational resources in anticipation of upcoming demands, in accordance with SLAs.

[0105] The workload layer 90 illustrates examples of functionality available in a cloud computing environment. Examples of workloads and functionality that may be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and secure matching and identification services 96. Secure matching and identification 96 is a service that implements the embodiments provided herein, such as implementing the operations of cloud service provider 320 of FIGS. 2-4.

[0106] The present invention may be a system, method, and / or computer program product, and may include a computer readable storage medium including computer readable program instructions for causing a processor to perform aspects of the present invention.

[0107] A computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridges in grooves on which the instructions are recorded, and any suitable combinations of these. As used herein, computer-readable storage media should not itself be construed as being ephemeral signals such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over wires.

[0108] The computer readable program instructions described herein can be downloaded from a computer readable storage medium to each computing device / processing device or to an external computer or storage device over a network (e.g., the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof) that may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing device / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions for storage on a computer readable storage medium within each computing device / processing device.

[0109] The computer readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++®, and traditional procedural programming languages ​​such as the “C” programming language® or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, to carry out aspects of the invention, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer readable program instructions to customize the electronic circuitry by utilizing state information of the computer readable program instructions.

[0110] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks included in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0111] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, where the instructions, executed by the processor of the computer or other programmable data processing apparatus, produce means for performing the functions / operations specified in the blocks of the flowcharts and / or block diagrams. These computer readable program instructions may be stored on a computer readable storage medium and capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions for performing aspects of the functions / operations specified in the blocks of the flowcharts and / or block diagrams.

[0112] Computer readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / acts specified in the blocks of the flowcharts and / or block diagrams, thereby causing a series of operable steps to be performed on the computer, other programmable apparatus, or other device to generate a computer-implemented process.

[0113] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously or in reverse order, depending on the functionality involved. It will also be noted that each block in the block diagrams and / or flowchart illustrations, as well as combinations of blocks included in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

Claims

1. A first computer system comparing an encrypted query entity representation with one or more encrypted gallery entity representations, wherein a comparison of the encrypted query entity representation with the one or more encrypted gallery entity representations results in a doubly encrypted value of an index between corresponding compared representations, the encrypted query entity representation being homomorphically encrypted using a homomorphic encryption scheme using a first private key and the one or more encrypted gallery entity representations being homomorphically encrypted using a homomorphic encryption scheme using a second private key, and the comparison being performed within the encrypted domain without decrypting either representation; said first computer system outputting a selected one or more of said one or more doubly encrypted values ​​of said index; A method comprising:

2. The method of claim 1, wherein the outputting includes outputting a single selected one of the one or more doubly encrypted values ​​of the index and a corresponding single selected doubly encrypted index.

3. The method according to claim 1, further comprising: said first computer system using a doubly encrypted threshold T; the first computer system changing a particular value of the permuted plurality of selected doubly encrypted values ​​to a selected value based on the index before outputting the particular value. The method of claim 1 further comprising:

4. The method of claim 1, wherein the outputting step further comprises outputting both particular values ​​of the permuted plurality of selected doubly encrypted values ​​and their corresponding doubly encrypted indexes. The method according to claim 3.

5. The method of claim 3, wherein the outputting further comprises outputting only a portion of certain values ​​from the sorted plurality of selected doubly encrypted values.

6. The method of claim 1, further comprising: the first computer system sorting the selected doubly encrypted values ​​based on the value; said outputting includes outputting a subset of the permuted plurality of selected doubly encrypted values. The method of claim 1.

7. The method of claim 1, wherein the step of outputting further comprises outputting both the subset of the plurality of selected doubly encrypted values ​​and their corresponding doubly encrypted indices. The method according to claim 6.

8. The method further includes the first computer system receiving a query from a second computer system; the query includes the representation of the encrypted query entity; said outputting said selected one of said doubly encrypted values ​​is performed from said first computer system to said second computer system; said outputting further comprises outputting, by said first computer system, said selected one of said one or more doubly encrypted values ​​of said index to a third computer system. The method of claim 1.

9. The method of claim 1 , wherein the comparing and outputting is performed on the representation of the encrypted query entity.

10. The method of claim 1 , wherein the representation of one or more encrypted gallery entities corresponds to an image containing a face.

11. The method of claim 1 , wherein the representation of one or more encrypted gallery entities corresponds to a feature vector of facial information in an image.

12. The method of claim 1 , wherein the metric comprises a distance metric based on a cosine distance.

13. The method of claim 1 , wherein the metric comprises a similarity metric.

14. transmitting, by a first computer system, a query including a representation of an encrypted query entity to a second computer system, wherein the representation of the encrypted query entity is homomorphically encrypted using a first homomorphic encryption scheme using a first private key; receiving, at the first computer system, from the second computer system, one or more doubly encrypted values ​​indicative of metric values ​​used by the second computer system to compare one or more encrypted gallery entity representations with the encrypted query entity representations, the one or more doubly encrypted values ​​being encrypted using both the first homomorphic encryption scheme and a second homomorphic encryption scheme using a second private key; decrypting, by the first computer system, the one or more doubly encrypted values ​​to produce corresponding one or more first partially decrypted values ​​that are decrypted using the first homomorphic encryption scheme; receiving, at the first computer system, one or more second partially decrypted values ​​from a third computer system, the one or more second partially decrypted values ​​corresponding to distinct values ​​of the doubly encrypted value but decrypted with the second homomorphic encryption scheme; performing, by the first computer system, a merge on respective values ​​of the corresponding first and second partially decrypted values ​​to form a plaintext value, the plaintext value providing a value corresponding to the index and indicating how well a corresponding representation of an encrypted gallery entity is deemed to match a representation of the encrypted query entity; A method comprising:

15. 15. The method of claim 14, further comprising transmitting, by the first computer system, a public key corresponding to the first homomorphic encryption scheme towards the second computer system.

16. The method of claim 14, wherein receiving one or more doubly encrypted values ​​at the first computer system further comprises receiving the doubly encrypted values ​​and their corresponding doubly encrypted indexes at the first computer system.

17. determining, based on the indication, a particular one of the plaintext values ​​as a match with the representation of the encrypted query entity, the representation of the query entity corresponding to an encrypted version of a gallery entity; decrypting the doubly encrypted index corresponding to the particular plaintext value using the first homomorphic encryption scheme to create a first partially decrypted index; sending, from the first computer system, a request for the first partially decrypted index and a cleartext gallery entity corresponding to the first partially decrypted index, to the third computer system; 20. The method of claim 16, further comprising:

18. 20. The method of claim 17, further comprising: receiving, at the first computer system, a plaintext gallery entity corresponding to a plaintext index from the third computer system; and determining whether the gallery entity corresponds to the plaintext gallery entity.

19. receiving, by a first computer system, from a second computer system, one or more doubly encrypted values ​​indicative of an index value used by the second computer system to compare one or more encrypted gallery entity representations with encrypted query entity representations, the one or more doubly encrypted values ​​being encrypted using both a first homomorphic encryption scheme using a first private key and a second homomorphic encryption scheme using a second private key; decrypting, by the first computer system, the one or more doubly encrypted values ​​to produce one or more corresponding first partially decrypted values ​​that are decrypted using the second homomorphic encryption scheme; outputting the one or more first partially decrypted values ​​to a third computer system, the one or more first partially decrypted values ​​being decrypted with the second homomorphic encryption scheme; A method comprising:

20. 20. The method of claim 19, further comprising transmitting, by the first computer system, a public key corresponding to the second homomorphic encryption scheme to the second computer system.

21. The method of claim 19, wherein receiving one or more doubly encrypted values ​​at the first computer system further comprises receiving the one or more doubly encrypted values ​​and their corresponding doubly encrypted indexes at the first computer system.

22. the first computer system partially decrypting a selected one of the doubly encrypted indexes using the second homomorphic encryption scheme to create a first partially decrypted index; receiving, by the first computer system, from the third computer system, a second partially decrypted index that corresponds to the first partially decrypted index, the second partially decrypted index having been partially decrypted using the first homomorphic encryption scheme; merging the first and second partially decrypted indexes to form a plaintext index; 22. The method of claim 21 further comprising:

23. 23. The method of claim 22, further comprising: the first computer system receiving a request from the third computer system for a plaintext gallery entity corresponding to the plaintext index; and in response to the request, transmitting the plaintext gallery entity from the first computer system to the third computer system.

24. A computer program product for causing a computer to carry out the steps of the method according to any one of claims 1 to 23.

25. A computer readable storage medium having a computer program recorded thereon, the computer program being for causing a computer to carry out each step of the method according to any one of claims 1 to 23.

Images