Security matching and identification of patterns

Abstract

A framework is provided in which a querying authority can request encrypted data from a data owning authority storing the encrypted data (via a query entity) through a service provider. The framework uses homomorphic encryption. The data can be gallery entities, and each element in the framework operates on doubly encrypted information. The service provider compares encrypted representations of the query entity from the querying authority and the gallery entities from the data owning authority, resulting in a doubly encrypted value of a metric between the corresponding compared representations. The querying authority obtains a result based on the metric that indicates whether the service provider is likely to have data similar or identical to the query data (305) in the query entity. These elements must communicate in order for the querying authority or the data owning authority to obtain plaintext information corresponding to the query entity.

Description

TECHNICAL FIELD

[0001] The present invention relates generally to pattern matching and encryption, and more specifically to secure matching and identification of patterns. BACKGROUND

[0002] As enterprises move their data to the cloud for efficient large-scale processing of their transactions, there is an increasing concern and regulation related to data privacy and security. Specifically, placing unencrypted data on the cloud can allow unauthorized access to the data by others. On the other hand, encrypting data presents its own challenges, as without the proper key, a service provider on the cloud and hosting the information has the ability to determine what is in the encrypted data. In other words, if an enterprise wants to search for information in encrypted data, the enterprise will have to perform the search itself unless the enterprise gives the proper key to the service provider (and thereby compromise security). SUMMARY

[0003] In an example embodiment, a method includes comparing a representation of an encrypted query entity to one or more representations of encrypted gallery entities. The comparison between the representation of the encrypted query entity and the one or more representations of encrypted gallery entities results in double-encrypted values of a metric between the respective compared representations. The representation of the encrypted query entity is homomorphically encrypted using a first key and the one or more representations of encrypted gallery entities are homomorphically encrypted using a second key, and the comparison is performed in an encrypted domain without decrypting the representations. The method further includes outputting selected one or more of the double-encrypted values of the metric.

[0004] An additional example embodiment is a computer system. The computer system includes one or more processors and one or more memories including computer program code. The one or more memories and the computer program code are configured to, with the one or more processors, cause the computer system to perform operations comprising: comparing a representation of an encrypted query entity to one or more representations of encrypted gallery entities, wherein the comparison between the representation of the encrypted query entity and the one or more representations of encrypted gallery entities results in double-encrypted values of a metric between the respective compared representations, wherein the representation of the encrypted query entity is homomorphically encrypted using a first key and the one or more representations of encrypted gallery entities are homomorphically encrypted using a second key, and the comparison is performed in an encrypted domain without decrypting the representations; and outputting selected one or more of the double-encrypted values of the metric.

[0005] Another example embodiment is a computer program product comprising a computer readable storage medium having program instructions embodied therewith. The program instructions can be executed by a computer system to cause the computer system to perform operations comprising: comparing a representation of an encrypted query entity to one or more representations of encrypted gallery entities, wherein the comparison between the representation of the encrypted query entity and the representations of the one or more encrypted gallery entities results in a double-encrypted value of a metric between the respective compared representations, wherein the representation of the encrypted query entity is homomorphically encrypted using a first key and the representations of the one or more encrypted gallery entities are homomorphically encrypted using a second key, and performing the comparison in an encrypted domain without decrypting the representations; and outputting a selected one or more of the one or more double-encrypted values of the metric.

[0006] Another example embodiment is a method. The method comprises sending, by a first computer system to a second computer system, a query comprising a representation of an encrypted query entity, wherein the representation of the encrypted query entity is homomorphically encrypted using a first homomorphic encryption scheme with a first key. The method comprises receiving, at the first computer system from the second computer system, one or more double-encrypted values that indicate values of a metric used by the second computer system to compare representations of one or more encrypted gallery entities to the representation of the encrypted query entity, wherein the one or more double-encrypted values are encrypted using both the first homomorphic encryption scheme and a second homomorphic encryption scheme with a second key. The method further comprises decrypting, by the first computer system, the one or more double-encrypted values to create respective one or more first partially decrypted values that are decrypted in the first homomorphic encryption scheme. The method also comprises receiving, at the first computer system from a third computer system, one or more second partially decrypted values, wherein the one or more second partially decrypted values correspond to individual ones of the double-encrypted values but are decrypted in the second homomorphic encryption scheme. The method additionally comprises performing, by the first computer system, a merge for individual ones of the respective first and second partially decrypted values to form a plaintext value that provides a value corresponding to the metric and that indicates how well a respective representation of an encrypted gallery entity is considered to match the representation of the encrypted query entity.

[0007] Another example embodiment is an example apparatus comprising one or more processors and one or more memories including computer program code. The one or more memories and the computer program code are configured to, with the one or more processors, cause the apparatus to perform operations comprising: sending, by a first computer system to a second computer system, a query comprising an encrypted representation of a query entity, wherein the encrypted representation of the query entity is homomorphically encrypted using a first homomorphic encryption scheme that utilizes a first key; receiving, at the first computer system from the second computer system, one or more doubly encrypted values indicative of values of a metric used by the second computer system to compare representations of one or more gallery entities to the encrypted representation of the query entity, wherein the one or more doubly encrypted values are encrypted using both the first homomorphic encryption scheme and a second homomorphic encryption scheme that utilizes a second key; decrypting, by the first computer system, the one or more doubly encrypted values to create a corresponding one or more first partially decrypted values, the first partially decrypted values being decrypted in the first homomorphic encryption scheme; receiving, at the first computer system from a third computer system, one or more second partially decrypted values, wherein the one or more second partially decrypted values correspond to individual ones of the doubly encrypted values but are decrypted in the second homomorphic encryption scheme; and performing, by the first computer system, a merge of individual ones of the corresponding first and second partially decrypted values to form a plaintext value that provides a value corresponding to the metric and that indicates how well a corresponding representation of an encrypted gallery entity is considered to match the representation of the encrypted query entity.

[0008] A computer program product is an additional example embodiment. The computer program product includes a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer system to cause the computer system to perform operations comprising: sending, by a first computer system to a second computer system, a query including a representation of an encrypted query entity, wherein the representation of the encrypted query entity is homomorphically encrypted using a first homomorphic encryption scheme that utilizes a first key; receiving, at the first computer system from the second computer system, one or more doubly encrypted values indicative of values of a metric used by the second computer system to compare representations of one or more encrypted gallery entities to the representation of the encrypted query entity, wherein the one or more doubly encrypted values are encrypted using both the first homomorphic encryption scheme and a second homomorphic encryption scheme that utilizes a second key; decrypting, by the first computer system, the one or more doubly encrypted values to create a respective one or more first partially decrypted values that are decrypted in the first homomorphic encryption scheme; receiving, at the first computer system from a third computer system, one or more second partially decrypted values, wherein the one or more second partially decrypted values correspond to individual ones of the doubly encrypted values but are decrypted in the second homomorphic encryption scheme; and performing, by the first computer system, a merge of individual ones of the respective first and second partially decrypted values to form a plaintext value that provides a value corresponding to the metric and that indicates how well a corresponding representation of an encrypted gallery entity is deemed to match the representation of the encrypted query entity.

[0009] Another example embodiment is a method that includes receiving, at a first computer system from a second computer system, one or more doubly encrypted values indicative of values of a metric used by the second computer system to compare representations of one or more encrypted gallery entities to a representation of an encrypted query entity. The one or more doubly encrypted values are encrypted using a first homomorphic encryption scheme and a second homomorphic encryption scheme, the first homomorphic encryption scheme using a first key and the second homomorphic encryption scheme using a second key. The method includes decrypting, by the first computer system, the one or more doubly encrypted values to create a respective one or more first partially decrypted values that are decrypted in the second homomorphic encryption scheme. The method also includes outputting, to a third computer system, the one or more first partially decrypted values that are decrypted in the second homomorphic encryption scheme.

[0010] Another example apparatus includes one or more processors and one or more memories including computer program code. The one or more memories and the computer program code are configured to, with the one or more processors, cause the apparatus to perform operations comprising: receiving, at a first computer system from a second computer system, one or more double-encrypted values indicative of values of metrics used by the second computer system to compare representations of one or more encrypted gallery entities to a representation of an encrypted query entity, wherein the one or more double-encrypted values are encrypted using both a first homomorphic encryption scheme that utilizes a first key and a second homomorphic encryption scheme that utilizes a second key; decrypting, by the first computer system, the one or more double-encrypted values to create one or more first partially-decrypted values that are decrypted in the second homomorphic encryption scheme; and outputting, to a third computer system, the one or more first partially-decrypted values that are decrypted in the second homomorphic encryption scheme.

[0011] A further example embodiment is a computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer system to cause the computer system to perform operations comprising: receiving, at a first computer system from a second computer system, one or more double-encrypted values indicative of values of metrics used by the second computer system to compare representations of one or more encrypted gallery entities to a representation of an encrypted query entity, wherein the one or more double-encrypted values are encrypted using both a first homomorphic encryption scheme that utilizes a first key and a second homomorphic encryption scheme that utilizes a second key; decrypting, by the first computer system, the one or more double-encrypted values to create one or more first partially-decrypted values that are decrypted in the second homomorphic encryption scheme; and outputting, to a third computer system, the one or more first partially-decrypted values that are decrypted in the second homomorphic encryption scheme. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 Matching and identification of plaintext of a gallery of face images is shown;

[0013] Figure 2 Matching and identification of a gallery of face images in an encrypted domain is shown;

[0014] Figure 3 Block diagram of an example system suitable for secure matching and identification of patterns according to example embodiments is shown;

[0015] Figure 3Ais a block diagram of another exemplary system suitable for secure matching and identification of patterns according to exemplary embodiments;

[0016] Figure 4 is a block diagram of a flowchart of an exemplary method for secure matching and identification of patterns according to exemplary embodiments;

[0017] Figure 5 a graphical illustration of a matching algorithm is provided according to exemplary embodiments;

[0018] Figure 6A is a graphical illustration of an algorithm for finding an index of a matching identity of homomorphically encrypted numbers;

[0019] Figure 6B is a graphical illustration of an algorithm for finding a square root of two homomorphically encrypted numbers;

[0020] Figure 6C is a graphical illustration of an algorithm for finding values of min(a, b) and max(a, b);

[0021] Figure 7 is a block diagram of one possible and non-limiting exemplary system in which exemplary embodiments can be implemented;

[0022] Figure 8 depicts a cloud computing environment according to exemplary embodiments; and

[0023] Figure 9 depicts an abstraction model layer according to exemplary embodiments. DETAILED DESCRIPTION

[0024] The following abbreviations which can be found in the specification and / or drawings, are defined as follows:

[0025] FHE fully homomorphic encryption

[0026] HE homomorphic encryption

[0027] I index

[0028] I / F interface

[0029] MFHE multi-key fully homomorphic encryption

[0030] SMC secure multi-party computation

[0031] SV similarity value

[0032] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations. All of the implementations described in the DETAILED DESCRIPTION are intended to be exemplary implementations provided to enable persons skilled in the art to make or use the application and not to limit the scope of the application defined by the claims.

[0033] As noted above, enterprises that move their data to the cloud (e.g., to efficiently process their transactions at scale) must address increasing concerns and regulations related to data privacy and security. The encryption toolset is one means of transmitting data to the cloud while providing specific guarantees of security and privacy. However, as noted previously, placing encrypted data on the cloud or similar storage location makes searching more difficult. In exemplary embodiments, the techniques provided below provide an end-to-end encryption solution to certain "matching" applications on the cloud or other suitable location.

[0034] For example, what an enterprise can want to do is perform matching and identification in plaintext (unencrypted information). Consider a useful class of applications called "identification" that involves the following:

[0035] 1) a pre-populated representation of one or more entities, collectively called a gallery;

[0036] 2) given a new entity representation (called a query or probe), the system needs to evaluate which of the gallery entities is "most similar" to the query entity;

[0037] 3) said evaluation includes whether the query entity is sufficiently similar to a gallery entity to determine that a "match" has been found; and

[0038] 4) the representation can be structured (e.g., "name" ss#) or unstructured (e.g., an image).

[0039] Example applications include facial recognition, license plate recognition, etc. With respect to facial recognition, Figure 1 A plaintext matching and identification of a gallery of facial images is shown. In this example, the database 110 can be on the cloud and the gallery 115 includes images of many faces, where images 1 through 18 are shown. The enterprise ends a query 130 (also called a probe) with a corresponding image 125 as well as a match request 120 (shown as "match"). For example, there can be facial security enabled to access employee information, and the enterprise wants to verify that the employee can access the employee information.

[0040] The evaluation is made using the following equation:

[0041] s(q,g i )> T, (1)

[0042] where:

[0043] q: a representation of a query entity;

[0044] g i : a representation of the ith gallery entity;

[0045] s: a similarity function; and

[0046] T: a threshold.

[0047] In this example, the result is "yes" as indicated by reference numeral 140 (i.e., there is a match), and also s(q, g5) > T (see reference numeral 145) because the fifth image (image 5) matches the input image 125 s(·) according to the above equation (1) and based on the similarity function. As indicated by reference numeral 150, other structured information (e.g., access level to indicate whether an employee has access to employee information) can be utilized to output image 5.

[0048] While this provides benefits to the enterprise, it also means that much personal data (in this example, particularly images of employees) is outside the control of the enterprise. In fact, with respect to the example of facial images, there has recently been privacy concerns over facial recognition. One concern involved a report indicating that certain government agencies were using databases for facial recognition searches. Specifically, government organizations were using driver's license photos. However, if an enterprise puts facial images as plaintext on the cloud or other location that can be accessed by anyone, the enterprise risks similar privacy issues.

[0049] Furthermore, many regulations have begun to address privacy issues and impose penalties or take other action for violations of the regulations. For example, as of May 25, 2018, the European Union (EU) General Data Protection Regulation (GDPR) went into effect and imposed heavy fines of up to 4% of annual revenue for violating organizations. Many companies operate in regulated industries and handle sensitive data, adding significant costs to operations.

[0050] In view of such issues, data can be encrypted and searches performed on the encrypted data. Figure 2 Matching and identification in the encrypted domain is shown. Reference is made to Figure 1 , Figure 2 The matching in

[0051] This is shown in Figure 2 , where the gallery 115 in plaintext (i.e., unencrypted) is encrypted 260 and becomes the encrypted gallery 215 in the database 110-1. The encryption function used for the encryption operation 260 is exemplified as E(g i , Kg ), where E(·) is an encryption function, g i is a representation of the ith gallery entity, and K g is a key used for the gallery and used by the encryption function to encrypt the images. The encrypted gallery 215 has encrypted images 1-1 through 18-1, each of which is an encrypted version of the corresponding plaintext image 1 through 18.

[0052] Using this system, the enterprise encrypts 255 the input image 125 to create an encrypted input image 125-1. The match request 120-1 includes an encrypted query (also referred to as a probe) 130-1 represented as E(q, K q ), where E(·) is an encryption function, q is a representation of the query entity (e.g., including the encrypted input image 125-1), and K q is a key used for the query and used by the encryption function to encrypt the query q into the encrypted query 130-1.

[0053] The result is an encrypted image 270 returned to the enterprise. This result is created using the following equation:

[0054] E(s(q, g i ), K g , K q )>E(T, K g , K q ), (2)

[0055] where:

[0056] q: a representation of the query entity;

[0057] g i : a representation of the ith gallery entity;

[0058] s: a similarity function;

[0059] T: a threshold;

[0060] E(·): an encryption function;

[0061] K g : a key used for the encrypted gallery; and

[0062] K q : a key used for the encrypted query.

[0063] In this example, the result is E(s(q, g5), K g , K q ) because the encrypted fifth image (image 5) 6-1 matches the encrypted input image 125-1 according to the above equation (2) and based on the similarity function s(·). The enterprise will use the key K qThe result 270 of the decryption 265 is used to obtain image 5, as shown by reference numeral 150.

[0064] There are two competing interests here: on the one hand, businesses want to control access to their data, particularly private data, such as by using encryption in Figure 2

[0065] Exemplary embodiments herein address these issues by providing a framework in which, for example, a querying agency that has a legitimate reason to request data can request encrypted data from a data owning agency that stores the encrypted data through a service provider. Figure 3 An exemplary system is shown that is suitable for pattern matching and identification of security according to exemplary embodiments. The framework allows a querying agency 310 to request a match to query data 305 from a service provider 320. The querying agency 310 gets a result, a similarity value 390, that indicates whether the service provider 320 can have data that is similar or identical to the query data 305. Additional verification 370 can be performed with the approval of the data owning agency 330. This is described in more detail below.

[0066] As an example, as mentioned previously, in this framework 300 the querying agency is allowed to make a query for a valid reason, such as to implement a warranty, to be in compliance with a measurement (e.g., according to a regulation), or for another legitimate reason. As one example, the querying agency 310 can be the Federal Bureau of Investigation (FBI), and the data owning agency can be another part of the federal government. The FBI can need to know whether a picture of a person they took is of a wanted criminal. They can use this limited query to determine whether the person in the picture is really a wanted criminal. Note that the verification in block 370 can pass (e.g., the facial image is similar) or fail (e.g., the facial image is not similar, such as from a different person).

[0067] Note that the service provider 320 can allow queries from certain organizations, or only so many queries in a period of time, or impose other restrictions on the querying agency. The data owning agency 330 can also provide approval as the case can be, or a court can provide approval for the queries. The restrictions and approvals are outside the scope of this disclosure. Instead, this disclosure provides the framework 300 for such (typically limited) queries to occur on encrypted data.

[0068] ​In more detail, three entities are involved in this framework: a querying authority 310; a service provider 320, which can be on the cloud; and a data owning authority 330. The querying authority 310 comprises an encryption process 308, a decryption process 330 and a merging process 350, and can comprise a verification process 370. The service provider 320 comprises a matching process 315. The data owning authority 330 comprises an encryption process 320, a decryption process 340 and a merging process 360. The data owning authority 330 owns data that is placed together with the service provider 320. The querying authority 310 wants to determine whether a query data 305 is contained within a (e.g. encrypted) database 389.

[0069] Figure 3 The operations taken by the intermediate entities are described by way of example in Figure 4 , which are best explained by way of example, Figure 4 is a block diagram of a flowchart of an exemplary method for secure matching and identification of patterns according to exemplary embodiments. Reference is made to Figure 3 and Figure 4 The following description is made with reference to the drawings, in which reference signs with 3XX (where "X" is any integer) are on Figure 3 (or on Figure 3A ) and reference signs with 4XX are on Figure 4 .

[0070] Note that current technology such as Figure 3 and 4 implement pattern matching within an encrypted domain using techniques of multi-key fully homomorphic encryption (MFHE). For these multi-key techniques, see Adriana López-Alt, Eran Tromer and Vinod Vaikuntanathan, "On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption", Proceedings of the Forty-Fourth Annual ACM Symposium on Theory of Computing, ACM, 2012.

[0071] At block 410, the data owning authority 330 uses the encryption process 320 to use a key K g326. Encrypt the representation of the data entity (e.g., gallery data 322). It should be noted that anything that can be executed in fully homomorphic mode can also be executed in slightly homomorphic mode, but not vice versa. The data owner 330 sends the obtained encrypted gallery data 316 to the service provider 320. The data owner 330 also sends the public key 302 to the service provider 320. Fully homomorphic technology creates the key K. g 326 and public key 302. Note that a gallery of face images is used as the primary example in this article, but the techniques described are not limited to this example. Furthermore, the term "gallery" can encompass a collection of objects or other entities in a database.

[0072] It should also be noted that these representations can be encrypted data of images in a gallery (for images). However, it is more likely that the representations are encrypted data of feature vectors of facial information, which can then enable better comparison. These representations can be received as such feature vectors (as encrypted gallery data 316), although encrypted gallery data 316 can be received as images and these images can be converted into feature vectors using homomorphic operations. That is, gallery data 316 is encrypted. Given an encrypted image, encrypted feature vectors can be computed in the homomorphic domain, making it possible to create feature vectors homomorphically from image or text input. The feature vectors can then be stored in the encrypted gallery 317 along with the corresponding image, for example, as metadata of the corresponding image. Similarly, encrypted query data 312 can also be encrypted images or encrypted feature vectors of images.

[0073] Service provider 320 places encrypted gallery data 316 into encrypted gallery 317 in database 389. (Reference) Figure 2 Encrypted library 317 can be considered as encrypted library 215, and database 389 can be considered as database 110-1. Encrypted library data 316 in encrypted library 317 can be considered as a pattern because the encrypted data is essentially a binary dataset that forms a pattern, which can then be compared with other patterns, as described below.

[0074] exist Figure 4 In box 420, the querying agency 310 uses encryption processing 308 using key K q 325 uses fully homomorphic techniques to encrypt the representation of the query entity (e.g., query data 305). For example... Figure 5 As shown, multiple query entities can exist in a query. The query agency 310 sends the obtained encrypted query data 312 as part of a query (also called a probe) 313 to the service provider 320. The query agency 310 also sends the public key 301 to the service provider 320. Fully homomorphic technology is used to create the key K. q Both 325 and public key 301.

[0075] In response to receiving the encrypted query data 312, the service provider 320 runs a matching process 315. In Figure 4 the matching process 315 includes the operations in blocks 430 and 440. In block 430, the service provider uses a metric (e.g., distance, cosine distance, similarity metric) and corresponding algorithm to perform comparisons to find the value of the metric between the encrypted query / probe 313 and each encrypted representation of entities in the gallery, where the resulting values are encrypted by K g and K q both. Figure 6A 、 Figure 6B and Figure 6C Exemplary algorithms that can be used are shown in the above-referenced articles. Note that the service provider 320 is able to perform distance calculations and compare the calculated distances to a threshold without accessing the secret keys K g and K q . In more detail, the SP (service provider) 320 has gallery data 316 encrypted under key K g and has query data 312 encrypted under key K q When two inputs to a calculation (such as a distance calculation) are encrypted using different keys, the output distance will be encrypted by both of the keys. One option to implement this property is to use the method of the multi-key homomorphic encryption scheme presented in Hao Chen et al., “Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference,” The 26th ACM Conference on Computer and Communications Security (CCS 2019), DOI: 10.1145 / 3319535.3363207. More specifically, as described in Section 4 of the above-referenced article, any operation between ciphertexts encrypted using different keys results in expanded ciphertext at the SP (service provider). The public keys 301 and 302 of each party (the querying authority and the data owning authority) can be combined with a corresponding evaluation key (which is a special encryption of each party’s secret key itself) to generate a relinearization key for each party. The expanded ciphertext sent by the SP can be transformed into more traditional ciphertext by both parties by applying a complex process called relinearization using their respective relinearization keys. These traditional ciphertexts can be partially decrypted by both parties and eventually combined to get the final decrypted result of the calculation.

[0076] Cosine distance is one exemplary distance metric that can be used. Note that the metric used effectively "ignores" that the data is encrypted. In other words, the metric does not need to be modified for use with encrypted data. It should also be noted that other "distance" metrics can be used. Many such metrics exist, such as the Hamming distance metric. However, different "similarity" metrics and corresponding algorithms can be used alternatively or additionally. These are similar but use different scales. For example, for distance metrics, the minimum distance between multiple items indicates the "best" match between items, while for similarity metrics, the highest similarity metric indicates the best match. Many other distance measures (e.g., Hamming, Euclidean, Manhattan, Mahalanobis, etc.) or similarity measures (e.g., correlation) can be used.

[0077] In box 440, service provider 320 determines which selected values ​​are modified and sends them to query agency 310 and data ownership agency 330. Boxes 430 and 440 are in... Figure 5 An exemplary embodiment is illustrated in the following description.

[0078] In progress Figure 4 Before proceeding with the additional description, it is helpful to describe possible exemplary processes performed by the service provider 320. First, consider... Figure 3 Only a single encryption result 397 (I) is shown in the image. qg ,SV qg ), where the subscript qg indicates the key K. q 325 and K g Double encryption of the homomorphic encryption scheme corresponding to 326 (e.g., or its corresponding public keys 301, 302). However, for the reasons described below, service provider 320 may alternatively produce multiple results 397. This is also due to... Figure 3A As shown, Figure 3A This is another exemplary system suitable for secure pattern matching and recognition, according to an exemplary embodiment. Figure 3 A block diagram. Figure 3 One such multiple result 397-1 is a vector of N indices and their corresponding N similarity values: [(I1,SV1),(I2,SV2),…,(I... N ,SV N Another example is multiple results 397-2, which includes a vector of N indices and a vector of N corresponding similarity values: [I1, I2, ..., I...]. N ] and [SV1,SV2,…,SV NThe indices are information that allows, for example, in the database 389, to uniquely access the corresponding similarity values with respect to other similarity values. The similarity values can also correspond to images (or feature vectors or gallery data 316 that is however encrypted is stored). Furthermore, the decryption in block 330 can comprise decrypting the N indices I g1…N and the N similarity values SV g1…N . The notation I g1…N is used to save space and is equivalent to (I g1 , I g2 ,..., I gN ). Similarly, the notation SV g1…N is used to save space and is equivalent to (SV g1 , SV g2 ,..., SV gN ). Similar notations are used elsewhere in Figure 3A . For example, the decryption 340 can decrypt can comprise decrypting the N indices I q1…N and the N similarity values SV q1…N .

[0079] Second, with respect to this process, it is important to understand that the service provider 320 operates entirely in the encrypted domain. Thus, while the service provider 320 can perform multiple computations, and run algorithms and use thresholds (as described below), the service provider does not know the results of these computations.

[0080] Consider some simple examples. Assume that the service provider 320 computes the vector of similarity values [SV1, SV2, SV3, SV4, SV5, SV6] after matching. Assume that the decision rule of step 440 (see block 311) is to send the first two similarity values that are greater than a (doubly encrypted) threshold T. The service provider can compute max(SV i -T, 0) (where max(A, B) selects the maximum between A and B), sort the results, and mask out entries except the first two. The decision rule 311 can use a limit, referred to herein as L. In examples 1 and 2 below, L = 2. The decision rule 311 and its limit L can be set by the data owner 330, the querying authority 310, and / or the service provider 320. The limit L can be in the clear, or a doubly encrypted value depending on the implementation.

[0081] Example 1 : Assume that SV2, SV4, and SV5 are greater than T, where SV2 > SV5 > SV4. In this case, the encrypted similarity vector can be [SV2, SV5, 0, 0, 0, 0], and the encrypted index vector will be [2, 5, 0, 0, 0, 0]. These two results will be sent as results 397 to the querying authority and the data owning authority, respectively. Note that the zeros ("0") are actually double-encrypted representations of zero. Also note that the values used will change based on the metric used, e.g., distance values can be ones ("1") instead of zeros.

[0082] Example 2: Assume that none of the similarity values are greater than T. In this case, the encrypted similarity vector will be [0, 0, 0, 0, 0, 0], and the encrypted index vector will be [0, 0, 0, 0, 0, 0]. These two results (the first two zeros) will be sent as results 397 to the querying authority and the data owning authority, respectively.

[0083] Note that in both examples, the service provider does not know what is inside the result vector. Only the querying authority and the data owning authority can jointly decrypt the results 397.

[0084] In the first example, the querying authority 310 will know the first two similarity values, and the data owning authority 330 will know the indices of the first two most similar matches. In the second example, both authorities will know that there are not enough similar matches.

[0085] Accordingly, in block 450, which is an example of block 440, the service provider 320 uses the double-encrypted threshold T to determine which encrypted query entities are sent as-is and which are modified (e.g., zeroed out), and outputs the encrypted results 397 to the querying authority 310 and the data owning authority 330. Note that the "double-encrypted threshold" is conceptual. In some implementations, the threshold can be explicitly encrypted using a public key and used as described herein. In other implementations, a plaintext threshold can be used, but the comparison results will still be double-encrypted. This is due to the nature of the FHE scheme.

[0086] In this example, the encrypted result 397 includes the index 318 (e.g., into the database 389) of the corresponding matching gallery entry as well as the corresponding similarity value 319. The similarity value is an indication of how “similar” the matching gallery entry is to the encrypted query data 312. For example, one common scale used is from zero (not similar) to one (very similar), and the similarity value will be within this range. Other ranges can be used as well. It should be noted that the document by Hao Chen et al. (“Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference”) also describes an exemplary prediction model that can be used as a suitable algorithm for “similarity” determination.

[0087] However, it is important to note that these are not “raw” similarity values, but rather doubly encrypted similarity values 319. The query authority 310 cannot decrypt them without help from the data owning authority 330 as well as the query authority 310. Furthermore, it is important to reiterate that the service provider 320 does not know whether any gallery entity is sufficiently similar to the query, since the service provider 320 only sees the doubly encrypted comparison result 397. Only the query authority 310 (with help from the data owning authority 330 and the query authority) can see the comparison result in plaintext. Thus, the decision to discard or retain the selected gallery entity is not made by the service provider 320. However, the service provider 320 does use an algorithm and potentially a doubly encrypted threshold T (or a limit L indicating the number of results to send, which can be used with the decision rule 311) to limit the number of results sent.

[0088] From this, while all results 397 can be sent, a certain subset of the encrypted results 397 can be sent. For example, assume there are one million images (or feature vectors of images) in the encrypted gallery 317. This can result in an encrypted result 397 containing one million entries, each with an index and a corresponding similarity value. Instead of sending the entire one million entries, a double-encrypted threshold T can be used to select which subset to send, possibly in combination with a limit, L (as in examples 1 and 2 above, where the limit L indicates how many to send) can be used. This is shown by block 455, where the similarity values can be ordered based on the metric being used. For example, if the metric is a similarity metric, the higher numbers are better. At the same time, if the metric is a distance metric, the lower numbers are better. The double-encrypted threshold T can be used as a cutoff to determine which values will be sent. For example, for a similarity metric, the similarity values above the double-encrypted threshold T can be sent, and the values below or equal to the double-encrypted threshold T will not be sent from the service provider 320 to the querying authority 310, for example, and also not to the data owning authority 330. Another example uses a limit L, such that the double-encrypted threshold T is used to determine which results will be modified or will not be modified, but the limit L and decision rules 311 are used to determine which and which not to send. For example, only the top 10 results can be sent (according to the limit L), and the double-encrypted threshold is applied to all results to determine which are modified (e.g., to zero) and which are not.

[0089] With respect to blocks 430 and 440, it is conceivable to parameterize the inputs and results, including the size of the output list, the threshold, the distance (or other metric) function to use. This would allow more control over the actions of the service provider 320. However, again note that the service provider 320 only has information in encrypted form (threshold, result list, query, etc.). This information can only be decrypted by the querying authority 310 or the data owning authority 330 (for information encrypted by only one of them), or by both the querying authority 310 and the data owning authority 330 (for double-encrypted or partially decrypted from double-encrypted information). An important part to note is that the service provider 320 cannot deduce any part of the results. Rather, the service provider 320 only applies the encrypted similarity, encrypted threshold, etc., but does not know how many results qualify based on the specified threshold.

[0090] In block 460, the double-encrypted results 397 will be decrypted by the data owning authority 330 and the querying authority 310. The double-encrypted results 397 are decrypted by the data owning authority 330 using the decryption process 340, which uses the key K g 326. This produces results 341, which include the partially decrypted index I q and the partially decrypted similarity value SV qThey are decrypted with respect to the key K g 326, but remain encrypted with respect to the key K q 325, which is the reason for the "g" subscript.

[0091] The doubly encrypted result 397 is decrypted by the querying authority 310 using a decryption process 330 that uses the key K q 325. This produces a result 331 that includes a partially decrypted index I g and a partially decrypted similarity value SV g . They are decrypted with respect to the key K q 325, but remain encrypted with respect to the key K g 326, which is the reason for the "g" subscript.

[0092] In block 470, the partially decrypted similarity value SV q 385 is sent from the data owning authority 330 to the querying authority 310, and the partially decrypted matching index I g 380 is sent from the querying authority 310 to the data owning authority 330.

[0093] In block 480, the singly encrypted results 380, 385 are decrypted by the querying authority and the data owning authority. Specifically, the partially decrypted matching index I g 380 is decrypted by the data owning authority 330 to create an index 375 of matching gallery entries, where the index 375 is in the clear. Similarly, the partially decrypted similarity value SV q 385 is decrypted by the querying authority 310 to create a similarity value 390 of the matching gallery entities, where the similarity value is in the clear.

[0094] As another example, the multiple values passed by the client (in this case, the querying authority 310) to this point reflect the client's choices in the threshold and candidate list size (e.g., defined by the limits L and decision rules 311) of the list of responses other than images. The candidate list size defines, for example, the size of the encrypted results 397-1, 397-2, according to the limits L. These would exist even in applications like search. The threshold and candidate list size are used by the service provider 320 to limit the number of responses and are meaningful to the querying authority 310. For example, the querying authority 310 can be interested in responses with a match greater than 80%. Only the top five results. Another querying authority can be interested in a threshold greater than 0.5 and a candidate list size of 100. These are applied after the full results of the encrypted matches are obtained.

[0095] In the multi-key FHE scheme used herein, double-encrypted ciphertexts cannot be decrypted using two successive decryption operations involving both keys 325, 326. Instead, the way to decrypt double-encrypted ciphertexts in multi-key FHE is to partially decrypt them using the two keys separately, and then combine the results of the two partial decryptions to obtain the plaintext. This is why the combining steps 350, 385 are necessary. See, e.g., Hao Chen et al., Section 4.3, “Distributed Decryption,” “Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference” (Cryptology ePrint Archive: Report 2019 / 524). Section 4.3 of this document describes one example of distributed decryption using two algorithms of partial decryption and combining. Similar or identical algorithms can be used for the decryption 330, 340 and combining 350, 360 steps herein.

[0096] At this point, the querying authority 310 can determine that there is a match to the query data 305 in the database 389, and also determine what the similarity value 390 of the match is. Note that the similarity value 390 can not indicate a match. For example, if a similarity metric is used, zero can indicate dissimilarity, and one can indicate similarity. Thus, a value of 0.15 can indicate that there is no match, while a value of 0.95 can indicate a match. Note that the querying authority 310 has only the similarity value 390, but not the index 375, and thus cannot access the actual gallery data 316 corresponding to that index. Furthermore, even if the querying authority 310 had the index 375 and could access the image from the encrypted gallery 317 corresponding to that index, the image is encrypted (as encrypted gallery data 316). Thus, the querying authority 310 gets only limited information about the face image.

[0097] As a result, as an optional operation, the querying authority 310 can request the matching gallery entity in plaintext from the data owning authority 330 for verification. See block 490. Once the querying authority determines that there is a match, the querying authority 310 can follow up with a request to the data owning authority 330 (or some other organization) to provide the plaintext data. For example, if the Federal Bureau of Investigation (FBI) finds a match to a database of another federal agency, the FBI can initiate a court or other process to obtain the plaintext data (of just the matching entity) from that agency. This is why it is important for the data owning authority 330 to know the index 375 of the match, so that when a subsequent request comes from the querying authority 310, the data owning authority 330 can retrieve the correct data. This process is illustrated in FIG. 4B. Figure 3The matching gallery entity 395 is shown sent from the data owner 330 to the querying authority 310 in plaintext. The querying authority 310 performs a verification process 370 to verify that the matching gallery entity 395 corresponds to the query data 305. This step requires the data owner 330 to perform this action, and an approval scheme for this is outside the scope of this disclosure.

[0098] Additional verification in block 370 can include (see Figure 3 block 371) sending by the querying authority 310 the partially decrypted matching index I g (or multiple indices) and a request for merging of the indices (I g , I q ) (or corresponding sets of indices) to create a plaintext index I 375 (or multiple of these) and a request for the (plaintext) gallery data 395 corresponding to this plaintext index. Note that this can include the querying authority 310 selecting the "matching" similarity values SV x (or sets of these values) and the corresponding indices Ix (or sets of these). The one or more matching similarity values are based on the values of the metric being used, such as a similarity or distance metric. For the above example 1, the querying authority obtains the following vectors (as result 397-2): similarity values of [SV2, SV5]; and indices of [2, 5]. The querying authority 310 can select only the similarity value of SV2 and the index 2 to send to the data owner 330. If allowed and desired for the querying authority 310, the authority can send both the similarity values [SV2, SV5] and the indices [2, 5] to the data owner 330. The data owner 330 will respond with the corresponding plaintext gallery data 322 with index 2 or indices 2 and 5 if appropriate.

[0099] Figure 3A Similar concepts are also shown, where the querying authority 310 can send a request to the data owner 330 for a single index (I g1 ), and the data owner 330 can send the corresponding plaintext index 375 and corresponding gallery data 395. However, in another example, the querying authority 310 can send a request to the data owner 330 for multiple indices (I g1…M ), where I g1…M refers to (I g1 ,..., I gM ) and M > 1, and the data owner 330 can send M corresponding plaintext indices 375 and M corresponding gallery data 395 to the querying authority 310.

[0100] Note that in this example, the encrypted gallery data 316 is considered to be images containing face information. While images are assumed herein, the framework 300 can be applied to other data, such as employee records, for example, so long as the records have defining characteristics such that they can be compared. In fact, as immediately described below, face information is typically compared using a feature vector, which is a mathematical representation of the features of a face in an image. Thus, the comparison can be between vectors (or even matrices), or can use anything that can be represented as a vector (or matrix). That is, the technique is general and not limited to face images. Face images are merely illustrative examples.

[0101] Turning to Figure 5 , this figure provides an illustration of the matching algorithm according to an example embodiment. This is an illustration of the operations in blocks 430 and 440 of Figure 4 . This illustrates the way in which the numbers are packed so that the calculations can be performed efficiently. There are many other ways to pack these numbers. Similarly, the query can be packed differently as well, but the distances can still be obtained as single vectors. Thus, Figure 5 are merely illustrative and not limiting. In the example of Figure 5 , the gallery portion 317-1 includes four image feature vectors 510-1 through 520-4. That is, each column is a feature vector and corresponds to face information from one image as well as a gallery index. In this example, the probe 313 illustrates multiple images as encrypted query data 312, with image feature vectors 530-1 through 530-4 corresponding to face information from four different images.

[0102] A distance vector 510 is created for each pair of feature vectors 520, 530. That is, there will be 16 distance vectors 510. This illustrates the operations in block 430. Note that this can be performed via vector multiplication or, if supported, matrix multiplication.

[0103] Two distance vectors a 510-1 and b 510-2 are shown, and the determination of the vector with the maximum value between these two is determined as follows (see also reference 520): Max(a,b) = 0.5*((a+b) + sqrt((a-b)^2), which is In this case, the result 530 is the distance vector b 510-2. This illustrates part of block 440 of Figure 4 to find the maximum similarity value.

[0104] To find the index of the matching identity using, for example, the equation of reference numeral 520, multiple algorithms are used. One such algorithm is Algorithm 6: MaxIdx, which is presented in the following reference: Cheon, Jung Hee, et al. "Numerical Methods for Comparison on Homomorphically Encrypted Numbers." IACR Cryptology ePrint Archive 2019 (2019): 417. Figure 6A is a diagram of an algorithm for finding the index of the matching identity, according to exemplary embodiments. Figure 6B and Figure 6C is an additional diagram of the algorithm from Cheon that can be used. Figure 6B is a diagram of an algorithm for finding the square root of two homomorphically encrypted numbers. Figure 6C is a diagram of an algorithm for finding the values of min(a, b) and max(a, b), where min(·) finds the minimum value and max(·) finds the maximum value. No modifications to these algorithms are needed except for the fact that the inputs are double-encrypted.

[0105] Turning to Figure 7 , this figure illustrates a block diagram of one possible and non-limiting exemplary system 700 that can implement exemplary embodiments. In Figure 7 , computer system 710 is in wired and / or wireless communication with wired and / or wireless network 797, and communicates with other computer systems 790 over network 797. Computer system 710 can access via one or more wired or wireless networks 797, can be implemented in cloud 795, and the like. Computer system 710 can alternatively or additionally be a server, but does not require a client-server relationship.

[0106] Computer system 710 includes one or more processors 720, one or more memories 725, one or more transceivers 730, one or more network (N / W) interfaces (I / F) 745, and user interface circuitry 765, interconnected through one or more buses 727. Each of the one or more transceivers 730 includes a receiver, Rx 732, and a transmitter, Tx 733. The one or more buses 727 can be address, data, and / or control buses, and can include any interconnection mechanism, such as a motherboard or series of lines on an integrated circuit, optical fiber(s) or other optical communication equipment, and the like. The one or more transceivers 730 are connected to one or more antennas 728 (if a wireless system is used). The one or more memories 725 include computer program code 723.

[0107] The computer system 710 includes a control module 740, which includes one or both of portions 740-1 and / or 740-2. The control module 740 performs operations described herein, such as implementing any of the operations of the query mechanism 310, the query mechanism 310, or the service provider 320 described herein. The control module 740 can be implemented in a variety of ways. The control module 740 can be implemented in hardware as control module 740-1, such as being implemented as part of the one or more processors 720. The control module 740-1 can also be implemented as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 740 can be implemented as control module 740-2, which is implemented as computer program code 723 and executed by the one or more processors 720. For example, the one or more memories 725 and the computer program code 723 can be configured, with the one or more processors 720, to cause the computer system 710 to perform one or more of the operations as described herein, in response to retrieval and execution of the computer program code 723. It should also be noted that the devices shown in the computer system 710 are not meant to be limiting and additional, different or fewer devices can be used.

[0108] The user interface circuitry 765 is in communication with one or more user interface elements 705, which can be integral with or external to the computer system 710 but which is coupled to the computer system 710. The interface elements 705 include one or more of: one or more cameras; one or more audio devices (such as microphones, speakers, etc.); one or more sensors (such as GPS sensors, fingerprint sensors, orientation sensors, etc.); one or more displays; and / or one or more keyboards. A display 711 is shown, which can be external to or internal to the computer system 710. This list is not exhaustive or limiting, and other different or fewer elements can be used. A user 701-1, in this example a human, can interact with the computer system 710, for example to cause the system 710 to take certain actions. These actions can also be caused by the computer system 710 in conjunction with or without the action of the user 701-1. The computer system 710 communicates with other computer systems 790 via one or more wired or wireless networks 797, via one or both of wired and wireless links 777 and 778. Additionally, if the computer system 710 is implemented in the cloud 795, a user 701-2 can interact with the computer system 710 via the wired and / or wireless networks 797 and using the computer system 790. In this example, the computer system 790 will have appropriate user I / F circuitry and user I / F elements (not shown) but similar to the user I / F elements 705.

[0109] One example is where computer system 710 is in the cloud 795 and thus is the cloud service provider 320. The other computer system 790 would then be the querying authority 310 and the data owning authority 330. The control module 740 would implement the matching process 315 and other operations previously described as being performed by the service provider 320. In another example, the computer system 710 would be one of the querying authority 310 or the service provider 320 and the other computer system 790 would be the service provider 320, e.g., as a server connected to the computer system 710 via the network 797.

[0110] If the service provider 320 is implemented in the cloud 395, Figure 8 and Figure 9 Additional information is provided for cloud implementation. Figure 8 A cloud computing environment is depicted in accordance with example embodiments, and Figure 9 An abstraction model layer is depicted in accordance with example embodiments.

[0111] Referring now Figure 8 , an illustrative cloud computing environment 50 is described. Some or all of the cloud computing environment 50 implementing any of the examples herein can be considered a computer system 110. In other words, the computer system 110 can be considered part of the cloud computing environment 50 implementing any of the examples herein. As shown, cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone 54A, desktop computer 54B, laptop computer 54C, and / or automobile computer system 54N can communicate. Nodes 10 can communicate with one another. They can be grouped (not shown) physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described hereinabove, or a combination thereof. This allows cloud computing environment 50 to offer infrastructure, platforms and / or software as services with Figure 8 The types of computing devices 54A-N shown in FIG. 54 are intended to be illustrative only and computing nodes 10 and cloud computing environment 50 can communicate with any type of computerized devices over any type of network and / or network addressable connection (e.g., using a web browser).

[0112] Referring now Figure 9 , a set of functional abstraction layers are shown provided by cloud computing environment 50 Figure 8 It should be understood that Figure 9 The components, layers, and functions shown are merely illustrative and embodiments of the application are not limited thereto. As described, the following layers and corresponding functions are provided:

[0113] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframes 61; RISC (Reduced Instruction Set Computer) architecture based servers 62; servers 63; blade servers 64; storage devices 65; and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0114] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0115] In one example, management layer 80 can provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 82 provide cost tracking as resources are utilized within the cloud computing environment, and billing or invoicing for consumption of these resources. In one example, these resources can include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management such that required service levels are met. Service Level Agreement (SLA) planning and fulfillment 85 provide pre-arrangement for, and procurement of, cloud computing resources for which a future requirement is anticipated in accordance with an SLA.

[0116] Workloads layer 90 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions which can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics processing 94; transaction processing 95; and security matching and identification services 96. Security matching and identification 96 is a service that implements examples provided herein, such as implementing operations of a cloud service provider 320. Figures 2-4

[0117] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0118] ​A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, semiconductor, or any other suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0119] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0120] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0121] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0122] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0123] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0124] The computer program product can have signal recorded thereon in a variety of forms. Such forms include, but are not limited to, magnetic training, magnetic tape, electrical training, electrical tape, and compact discs. Therefore, the computer program product can take many forms.

Claims

1. A method comprising: comparing an encrypted representation of a query entity to one or more encrypted representations of gallery entities, wherein the comparison between the encrypted representation of the query entity and the one or more encrypted representations of gallery entities results in a double-encrypted value of a metric between the respective compared representations, wherein the encrypted representation of the query entity is encrypted using a first homomorphic encryption scheme with a first secret key and the one or more encrypted representations of gallery entities are encrypted using a second homomorphic encryption scheme with a second secret key, the double-encrypted value is jointly encrypted via both the first homomorphic encryption scheme and the second homomorphic encryption scheme, and the comparison is performed in the encrypted domain without decrypting the representations; and outputting selected one or more of the one or more double-encrypted values of the metric.

2. The method of claim 1, wherein, There is a single selected double-encrypted value of the one or more double-encrypted values of the metric that is outputted and a corresponding single selected double-encrypted index.

3. The method of claim 1, wherein there are multiple selected double-encrypted values and the method further comprises ordering the multiple selected double-encrypted values based on the values and modifying the selected values based on the metric using a double-encrypted threshold T before outputting certain of the ordered multiple selected double-encrypted values.

4. The method of claim 3, wherein, There are double-encrypted indices for corresponding representations of the multiple selected double-encrypted values and the output further comprises outputting both certain of the ordered multiple selected double-encrypted values and their corresponding double-encrypted indices.

5. The method of claim 3, wherein, The output further comprises outputting only some of certain of the ordered multiple selected double-encrypted values.

6. The method of claim 1, wherein there are multiple selected double-encrypted values, the method further comprises ordering the multiple selected double-encrypted values based on the values, and wherein the outputting comprises outputting a subset of the ordered multiple selected double-encrypted values.

7. The method of claim 6, wherein, There are double-encrypted indices for corresponding representations of the multiple selected double-encrypted values and the output further comprises outputting both the subset of the multiple selected double-encrypted values and their corresponding double-encrypted indices.

8. The method of claim 1, wherein: the method is performed by a first computer system; the method further comprises receiving a query from a second computer system, the query including the representation of the encrypted query entity; the outputting of the selected double-encrypted value of the double-encrypted values is performed from the first computer system to the second computer system; and the outputting further comprises outputting, by the first computer system, the selected double-encrypted value of the one or more double-encrypted values of the metric to a third computer system.

9. The method of claim 1, wherein a plurality of encrypted representations of a query entity are received in a query and the comparison and outputting are performed for the plurality of representations of the encrypted query entity.

10. The method of claim 1, wherein, The representations of one or more encrypted gallery entities correspond to images that include faces.

11. The method of claim 1, wherein, The representation of the one or more encrypted gallery entities corresponds to a feature vector of facial information in the image.

12. The method of claim 1, wherein the metric comprises a distance metric based on a cosine distance.

13. The method of claim 1, wherein the metric comprises a similarity metric.

14. A method comprising: sending, by a first computer system to a second computer system, a query comprising a representation of an encrypted query entity, wherein the representation of the encrypted query entity is homomorphically encrypted using a first homomorphic encryption scheme that utilizes a first key; receiving, at the first computer system from the second computer system, one or more doubly encrypted values indicative of values of a metric used by the second computer system to compare representations of one or more encrypted gallery entities to the representation of the encrypted query entity, wherein the one or more doubly encrypted values are encrypted using both the first homomorphic encryption scheme and a second homomorphic encryption scheme that utilizes a second key; decrypting, by the first computer system, the one or more doubly encrypted values to create a corresponding one or more first partially decrypted values, the first partially decrypted values being decrypted in the first homomorphic encryption scheme; receiving, at the first computer system from a third computer system, one or more second partially decrypted values, wherein the one or more second partially decrypted values correspond to individual ones of the doubly encrypted values but are decrypted in the second homomorphic encryption scheme; and performing, by the first computer system, a merge for individual ones of the corresponding first and second partially decrypted values to form plaintext values that provide values corresponding to the metric and that are indicative of a degree to which a corresponding representation of an encrypted gallery entity is deemed to match the representation of the encrypted query entity.

15. The method of claim 14, further comprising sending, by the first computer system to the second computer system, a public key corresponding to the first homomorphic encryption scheme.

16. The method of claim 14, wherein, There is a doubly encrypted index for the corresponding representation of the doubly encrypted values, and receiving the one or more doubly encrypted values at the first computer system further comprises receiving the doubly encrypted values at the first computer system along with their corresponding doubly encrypted indices.

17. The method of claim 16, further comprising: deeming a given one of the plaintext values to be a match to the representation of the encrypted query entity based on the metric, wherein the representation of the query entity corresponds to an encrypted version of a gallery entity; decrypting, using the first homomorphic encryption scheme, a doubly encrypted index corresponding to the given plaintext value to create a first partially decrypted index; and sending, from the first computer system to the third computer system, the first partially decrypted index and a request for the plaintext gallery entity corresponding to the first partially decrypted index.

18. The method of claim 17, further comprising receiving, at the first computer system from a third computer system, a plaintext gallery entity corresponding to the first partially decrypted index, and determining whether the gallery entity corresponds to the plaintext gallery entity.

19. A method comprising: receiving, at a first computer system from a second computer system, one or more double-encrypted values indicative of values of a metric used by the second computer system to compare a representation of one or more encrypted gallery entities to a representation of an encrypted query entity, wherein the one or more double-encrypted values are encrypted using both a first homomorphic encryption scheme that utilizes a first secret key and a second homomorphic encryption scheme that utilizes a second secret key; decrypting, by the first computer system, the one or more double-encrypted values to create a corresponding one or more first partially decrypted values that are decrypted in the second homomorphic encryption scheme; and outputting, to a third computer system, the one or more first partially decrypted values that are decrypted in the second homomorphic encryption scheme.

20. The method of claim 19, further comprising sending, by the first computer system to the second computer system, a public key corresponding to the second homomorphic encryption scheme. There is a double-encrypted index for corresponding representations of the double-encrypted values, and receiving one or more double-encrypted values at the first computer system further comprises receiving the one or more double-encrypted values and their corresponding double-encrypted index at the first computer system.

21. The method of claim 19, wherein, 22. The method of claim 21, further comprising: partially decrypting, by the first computer system, using the second homomorphic encryption scheme, a selected one of the double-encrypted indices to create a first partially decrypted index; receiving, by the first computer system from the third computer system, a second partially decrypted index that has been partially decrypted using the first homomorphic encryption scheme and corresponds to the first partially decrypted index; and merging the first partially decrypted index and the second partially decrypted index to form a plaintext index.

23. The method of claim 22, further comprising receiving, by the first computer system from the third computer system, a request for a plaintext gallery entity corresponding to the plaintext index, and sending, from the first computer system to the third computer system, the plaintext gallery entity in response to the request.

24. An apparatus comprising one or more processors and one or more memories including computer program code; the one or more memories and the computer program code configured to, with the one or more processors, cause the apparatus to perform the steps of a method according to any of claims 1-23. ​ ​ 25. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer system to cause the computer system to perform the steps of the method according to any one of claims 1 to 23.

Images