Fuzzy Data Matching Using Homomorphic Encryption

Abstract

A computer-implemented method and system (DMS) for facilitating data similarity queries across a network (CN) of data memories (DM1, DM2). The disclosed method and system is configured to probabilistically match data items held in the data memories (DM1, DM2) with cryptographic protection of the data items. The data matching method and system (DMS) is robust to discrepancies (typographical errors, minor discrepancies, etc.) in the matched data within a certain predetermined similarity threshold (q), typically described as a percentage. The disclosed method and system (DMS) enables fast, low-latency turnaround by aggregating and matching data items into data structures (M1, M2) that facilitate group-wise matching as opposed to pair-wise matching.

Description

[Technical field]

[0001] FIELD OF THEINVENTION The present invention relates to computer-implemented methods for facilitating data similarity queries across a network of data memories, systems for facilitating data similarity queries across a network of data memories, corresponding computer programs, non-transitory program storage media storing such programs, and computing devices or systems for executing the programs, as well as respective systems comprising one or more electronic data storage devices and the aforementioned computing devices. [Background technology]

[0002] Technical background If there is one resource that is not scarce in today's world, it is data. According to a June 2013 New York Times report ("How the U.S. Uses Technology to Mine More Data, Faster"), network-interconnected devices generate about 2.5 quintillion bytes of data per day. This is probably a hopeless underestimate for 2020, as the amount has grown substantially since then. And this amount of data is set to grow even more as many countries update their communications infrastructures to allow high-performance networks such as 5G for the Internet of Things (IoT) to connect even more players, tens of billions of devices with 24-hour data proliferation.

[0003] Managing this data overload is no mean feat: already today, as reported for example in Nature, Volume 561, Pages 163-166, 12 September 2018, data centers around the world consume an estimated 200 TWh (terawatt-hours) of energy each year.

[0004] Managing the resources to process this data is a challenge: appropriate technical tasks to make it more relevant include shielding devices from malware, managing network data traffic, low-latency information retrieval, and optimizing data storage.

[0005] Capacity optimization includes, for example, techniques for improving the overall storage capacity of a network data memory. Data compression or data re-duplication can be used. Data re-duplication is useful because data is not always cumulative and contains certain redundancies that one may wish to identify through a matching operation and then remove to improve data capacity. Similar data matching techniques can be used for bandwidth optimization in data exchange between networks.

[0006] Some of the data is personal data. For example, the data being processed may relate to confidential information that the data owner does not want to be leaked to other parties. In many jurisdictions, privacy laws have been put into place to protect the rights of data owners.

[0007] The requirement to respect the privacy of data owners makes managing data processing infrastructures even more difficult, especially with regard to the optimization of storage and network capacity by third parties for privacy reasons.

[0008] Although cryptographic techniques exist for encrypting sensitive data, techniques that scan data for information, such as data matching operations, cannot be used in this manner because encrypted data is typically unsuitable for such data processing.

[0009] Homomorphic encryption techniques have emerged that support direct operations on encrypted data. However, it has been found that using such homomorphic encryption schemes can add additional computational overhead that can result in very poor turnaround times.

[0010] Therefore, there may be a need for improved processing of encrypted data. The present invention may be used for any data processing of encrypted data that relies on robust data matching operations.

[0011] Aspects, examples and exemplary steps of the invention and embodiments thereof are disclosed below. The various exemplary features of the invention can be combined in accordance with the invention where technically suitable and practicable. Summary of the Invention [Means for solving the problem]

[0012] Illustrative Brief Description of the Invention Below, a brief description of certain features of the invention is given, which should not be understood to limit the invention to only the features or combinations of features described in this section.

[0013] The disclosed method and system are configured to probabilistically match data items held in data memories across a network with cryptographic protection of the data items. The data matching method and system are robust to discrepancies in the matched data (typographical errors, minor discrepancies, etc.) within a certain pre-defined similarity threshold, usually described as a percentage. The disclosed method and system enable fast, low-latency turnaround by aggregating and matching data items into a data structure that facilitates group-wise matching as opposed to pair-wise matching.

[0014] The system and method can be used, for example, to match patient records, including demographic records, genetic data such as genomic sequence data, and medical imaging data such as CT images, across or within a healthcare facility. However, the proposed system and method are not limited to the healthcare field and may be applied to a host of other technical applications such as data network traffic management, malware detection, distributed information retrieval, network capacity optimization, etc.

[0015] General description of the invention In this section, a description of general features of the invention is given, for example, by reference to possible embodiments of the invention.

[0016] In general, the present invention addresses the aforementioned needs in a first aspect by providing a computer-implemented cryptographic method for facilitating data similarity queries across a network of data memories, the method comprising: a) receiving from a first memory in the network a first ciphertext for a first aggregation of a first encoding of a first group of data items; b) receiving from a second memory in the network a second ciphertext for a second aggregation of the at least one second encoding of the at least one data item of the second group; and c) processing the first and second ciphertexts into a third ciphertext decipherable into a plaintext output, the plaintext output indicating or convertible to a similarity measure between data items from the two groups.

[0017] Instead of operating pairwise, the proposed method operates groupwise to achieve high throughput, low latency and efficient in-memory representation in the computing entity that executes the method. Preferably, the computing entity, a computing device such as a server, includes a processor of a multi-core design or other architecture that allows parallel execution and implementation, such as a GPU.

[0018] The aggregation of the first and second encodings is a respective data structure that supports efficient processing with low communication overhead. The aggregation allows the entirety of each of the first and second encodings to be addressed as a respective single entity. The aggregation is the result of aggregating the encodings into said data structure. A matrix is ​​an example of such an aggregation.

[0019] A data similarity query is a matching operation. The similarity queries primarily envisioned herein are membership queries, i.e. queries as to whether one or more data items from a first group are sufficiently similar to one or more data items of a second group.

[0020] The method includes performing, on at least one processor of at least one computing device, the following exemplary steps executed by the at least one processor:

[0021] In one example of the method according to the first aspect, the first encoding and / or the second encoding is a binary encoding.

[0022] In one example of the method according to the first aspect, the first and / or second encoding is by representation in at least one probabilistic data structure PDS.

[0023] In one example of the method according to the first aspect, the PDS includes any one or more of a Bloom filter, a Cuckoo filter, a quotient filter, a Morton filter, or an XOR filter.

[0024] In one example of the method according to the first aspect, the first and second aggregations are pre-obtained by forming respective matrices from said first encoding and at least one second encoding.

[0025] In one example of the method according to the first aspect, at least the first aggregation is formed by incorporating a first encoding into a row or column of a respective matrix.

[0026] In one example of the method according to the first aspect, operation c) comprises a matrix multiplication of at least two matrices.

[0027] In one example of the method according to the first aspect, operation c) is performed in a single instruction multiple data SIMD setting.

[0028] In one example of the method according to the first aspect, the plaintext output is represented as or convertible into a matrix.

[0029] In one example of the method according to the first aspect, the first and second ciphertexts are pre-obtained by respective homomorphic encryption operations. Preferably, the homomorphic encryption operations support plaintext linear combination operations including dot products or matrix multiplications.

[0030] In one example of the method according to the first aspect, the method includes decrypting the third ciphertext into a plaintext output.

[0031] In one example of the method according to the first aspect, the plaintext output represents a partial similarity measure that can be converted into said (full) similarity measure.

[0032] In one example of the method according to the first aspect, the similarity measure or the partial similarity measure is based on any one of, but not limited to, the Dice coefficient measure, the Tversky coefficient measure, and the Jaccard coefficient measure.

[0033] Depending on the similarity measure used, more or less additional information may need to be leaked. This is especially the case for Dice or Jaccard coefficient measures. Since homomorphic encryption does not generally support division, for decryption it may be necessary to expose two result ciphertexts, rather than just one (the third ciphertext generated in processing step c). In that case, the result may need to be decrypted and divided by the plaintext. Systems based on the Tversky exponent measure as preferably envisaged herein may in embodiments only need to provide said third ciphertext, and in some embodiments may reveal the similarity estimate upon decryption.

[0034] In one example of the method according to the first aspect, the method includes converting said plaintext output into said similarity measure.

[0035] In some embodiments, a similarity measure is used, such as the Dice or Tversky measures mentioned above or other similarity measures. Preferably, as envisaged herein, the similarity measure is based on discrete counts in the union and intersection sets. The union and intersection counts are obtained by aligning and comparing the encodings. The plaintext output facilitates the computation of such a similarity measure. Specifically, in an embodiment, the plaintext output, or the third ciphertext, represents a partial similarity measure, such as the intersection count. In an embodiment, the plaintext output, or the third ciphertext, forms intermediate data. In a transformation, the intermediate data is combined with auxiliary data, such as the union count, to fully reveal the similarity measure. In an embodiment, the transformation includes forming a linear combination of the partial measure (the intersection count) and the auxiliary data (the union count). The linear combination can be based on a similarity threshold associated with the similarity measure. The threshold represents a lower limit of similarity for data items to be classified as a match or mismatch. For example, a similarity threshold of 80% means that data items must be at least 80% similar to be classified as a match.

[0036] In one example of the method according to the first aspect, the first and / or second ciphertexts are pre-obtained by a packing operation followed by a homomorphic encryption operation.

[0037] A packing operation refers to a pre-processing that allows multiple values ​​to be encrypted into one ciphertext for the purposes of SIMD operations. In an embodiment, packing involves representing an aggregated data structure (such as a matrix) as a polynomial.

[0038] In a further aspect, a method for facilitating data similarity queries across a network of data memories is provided, the method comprising: a) forming i) a first aggregation from a first encoding of a first group of data items in a first memory, and ii) a second aggregation from at least one second encoding of at least one data item of a second group in a second memory; b) homomorphically encrypting the two aggregates into two ciphertexts.

[0039] Preferably, homomorphic encryption b) of the two aggregates results in two respective separate such ciphertexts, i.e., a single ciphertext for each aggregate. This allows and facilitates more efficient processing and lower communication overhead. The packing operation described above facilitates such encryption into respective single ciphertexts.

[0040] In a further aspect, a method is provided for facilitating data similarity queries across a network of data memories, the method comprising: decrypting a third ciphertext, previously obtained by processing a first and a second ciphertext, into a plaintext output convertible to a similarity measure of data items across a group, the first ciphertext being previously obtained by homomorphic encryption of a first aggregation of a first encoding of a first group of data items in a first memory, and the second ciphertext being previously obtained by homomorphic encryption of a first aggregation of a first encoding of a first group of data items in a first memory. of 2 and decrypting, previously obtained by homomorphic encryption, a second aggregation of the second encoding of at least one data item of the second group in the memory.

[0041] In a further aspect, a system for facilitating data similarity queries across a network of data memories is provided, the system comprising: a) receiving from a first memory in the network a first ciphertext for a first aggregation of a first encoding of a first group of data items; b) receiving from a second memory in the network a second ciphertext for a second aggregation of the at least one second encoding of the at least one data item of the second group; c) configured to process the first and second ciphertexts into a third ciphertext decipherable into a plaintext output, the plaintext output indicating or convertible into a similarity between data items from the two groups.

[0042] In a further aspect, a system for facilitating data similarity queries across a network of data memories is provided, the system comprising: a) forming i) a first aggregation from a first encoding of a first group of data items in a first memory, and ii) a second aggregation from at least one second encoding of at least one data item of a second group in a second memory; b) It is constructed to homomorphically encrypt the two aggregates into two ciphertexts.

[0043] In a further aspect, a system for facilitating data similarity queries across a network of data memories is provided, the system comprising: configured to decrypt a third ciphertext, previously obtained by processing the first and second ciphertexts, into a plaintext output convertible into a similarity measure of the data items across the group, the first ciphertext being previously obtained by homomorphic encryption of a first aggregation of a first encoding of the data items of the first group in the first memory, and the second ciphertext being previously obtained by homomorphic encryption of a second aggregation of a second encoding of at least one data item of the second group in the second memory.

[0044] As described above, and in accordance with some examples of the proposed methods and systems, Bloom filters are used to encode data items, such as records, such that the encoding is resistant to typographical errors. The Bloom filter encodings (preferably one per record) are then aggregated and "stacked" together to produce a Bloom filter matrix as an example of an aggregated data structure. This matrix is ​​then homomorphically encrypted into a single ciphertext, rather than a ciphertext per Bloom filter. This allows for group-wise, i.e. batch comparison of data items, reducing communication overhead and computational complexity quadratically.

[0045] The proposed system and method disclosed herein allows for efficient processing of encrypted data for data matching across data memories in a network while being robust to data variations. Data matches such as duplicates can be identified despite certain variations in the defined similarity threshold. The proposed system can be used in any kind of data processing application, which is not limited to text data, but can also include image processing applications, network and data storage management, etc. One application is the medical field. The medical field is a large data generator, where patient data is constantly increasing in the face of a growing sick population. Medical data includes patient records, written reports, image data, and measurements collected from medical sensors from IoT medical devices, such as biodata generated by smartphones, fitness trackers, and other wearable devices. This proliferation of medical data needs to be stored and managed. Patient medical data may be held in different facilities, one facility is unaware of the relevance held in another facility, and especially due to privacy laws, clinical facilities are prohibited from leaking data without authorization. The ability to find medical data from one data source, i.e. a single patient, held across multiple hospitals or medical or non-medical facilities (possibly distributed geographically, across a country or even internationally) can help build an accurate picture of a patient's medical condition and medical history.

[0046] In another aspect, the invention relates to a computer program which, when executed on at least one processor (e.g. one processor) of at least one computer (e.g. one computer) or when loaded into at least one memory (e.g. one memory) of at least one computer (e.g. one computer), causes at least one computer to perform any one of the above-mentioned methods. The invention may alternatively or additionally relate to a program, e.g. a (physical, e.g. electrical, e.g. technically generated) signal wave, e.g. a digital signal wave, carrying information representing said program, e.g. comprising code means adapted to perform any or all of the steps of the method according to the first aspect. The computer program stored on a disk is a data file, and when the file is read and transmitted, it becomes a data stream, e.g. in the form of a (physical, e.g. electrical, e.g. technically generated) signal. The signal may be implemented as a signal wave as described herein. For example, the signal, e.g. the signal wave, is configured to be transmitted over a computer network, e.g. a LAN, a WLAN, a WAN, e.g. the Internet. Thus, the invention according to the second aspect may alternatively or additionally relate to a data stream representing said program.

[0047] In a further aspect, the invention relates to a non-transitory computer readable program storage medium having stored thereon a computer program according to the previous aspect.

[0048] In a further aspect, the present invention relates to at least one computing device (e.g., a computer) comprising at least one processor (e.g., a processor) and at least one memory (e.g., a memory), wherein a computer program is executed on the processor or loaded into the memory, or wherein the at least one computer comprises a computer-readable program storage medium.

[0049] definition The systems and methods proposed herein operate in the plaintext domain and / or the ciphertext domain, or partly in both.

[0050] "Encryption" is a cryptographic mapping of plaintext in the plaintext domain to ciphertext in the ciphertext domain, denoted as x(plaintext) → {x}(ciphertext of x). The result of encryption is ciphertext.

[0051] "Decryption" is the inverse of encryption. Decryption is a cryptographic mapping from ciphertext in the ciphertext domain to plaintext in the plaintext domain. The result of decryption is plaintext. Plaintext can be recovered from its ciphertext by decryption only in combination with additional information, referred to herein as "key(s)". Without knowing the key, encryption cannot be performed in finite time with finite computational resources. The probability of achieving the inverse without the key is arbitrarily small. If x is the plaintext, then {x} represents its ciphertext.

[0052] "Ciphertext" does not leak the information content of the plaintext. The following notation is used herein: "{M}" is the ciphertext of plaintext "M", i.e., the curly brackets indicate the encrypted content.

[0053] An "encoding" is the result of an encoder's operation. An encoder (also referred to herein as a "filter") is a computational entity that maps plaintext into a combination of characters (one or more) from a given set, the "alphabet." That combination is an encoding for some data input. A binary encoder is one example used herein, where the alphabet contains only two characters, commonly called binary "0" and "1." Encoding is not encryption.

[0054] A "probabilistic data structure" ("PDS") is a data structure for representing or facilitating encoding and is contemplated herein in some embodiments. A Bloom filter is an example of a PDS. Some PDSs compactly represent a group or set of data items as a digest, such as using a hash function. PDSs approximately support operations such as membership queries, i.e., the result is "true" with a certain probability and / or an error bound is known on the result.

[0055] "Memory" as used herein may include memory portions of the same memory system, or may relate to different memories in different systems, possibly separate from one another.

[0056] A "network" includes an "internal" network where the two memories are not necessarily remote from each other (an "external network").

[0057] A "matrix" is a data structure that requires at least a 2D (two-dimensional) address to access its components. Said addresses may also be referred to herein as "index(s)". Matrices "tensors" of dimension n greater than 2 are also envisaged herein, and the addresses required are of dimension n (>2). A 2D matrix is ​​"n by m" in "size", where n, m are natural numbers. n, m relate to the rows and columns in which the components are organised in the 2D case. "mij" is shorthand for the components of a 2D matrix M, while mijk is shorthand for a 3D matrix M ("tensor"), and so on. A "vector" is a special case of a matrix where at least one of n or m is 1.

[0058] "Matrix multiplication", denoted herein as "*", is an operation or mapping that converts an "n x m" matrix and an "m x l" matrix into an "n x l" matrix.

[0059] "Transpose" is another matrix operation that transforms an "n x m" matrix into an "m x n" matrix. "Being transposed" is the act of achieving a transpose or refers to the state in which two matrices are transposed with respect to one another. Two matrices may be natively transposed without an explicit transpose operation if they are so generated. Two matrices may be natively transposed with respect to one another if, given first and second data, the first data is placed in the rows (columns) of one matrix and the second data is placed in the columns (rows) of the other matrix.

[0060] "1D", "2D", "3D", and "nD" are shorthand for n-dimension, n=1, 2, 3, ..., and refer to data arrangement.

[0061] A "computer-implemented method" as the term is used herein includes that all steps of a method according to the invention, or just a portion of the steps (i.e., less than the total number of steps), may be performed by a computer (e.g., at least one computer). One embodiment of a computer-implemented method is the use of a computer to perform a data processing method. One embodiment of a computer-implemented method is a method relating to the operation of a computer, such that the computer is operative to perform one, several, or all steps of the method.

[0062] A computer comprises, for example, at least one processor and, for example, at least one memory, for (technically) processing data, for example electronically and / or optically. The processor is, for example, made of a substance or composition that is, for example, a semiconductor, for example at least partially, n-type and / or p-type doped semiconductor, for example at least one of type II, III, IV, V, VI semiconductor materials, for example (doped) silicon and / or gallium arsenide. The described calculation or determination steps are, for example, performed by a computer. A determining step or a calculating step is, for example, a step of determining data within the framework of a technical method, for example within the framework of a program. A computer is, for example, any kind of data processing device, for example an electronic data processing device. A computer can be a device generally considered as such, for example a desktop PC, a notebook, a netbook, etc., but can also be, for example, any programmable device, such as a mobile phone or an embedded processor. A computer can, for example, include a system (network) of "sub-computers", each sub-computer representing a computer in itself. The term "computer" includes cloud computers, for example cloud servers. The term "cloud computer" includes cloud computer systems, including at least one system of cloud computers and a plurality of operatively interconnected cloud computers, such as server farms. Such cloud computers are preferably connected to a wide area network, such as the World Wide Web (WWW), and are located within a so-called cloud of computers, all connected to the World Wide Web. Such an infrastructure is used in "cloud computing" to describe computation, software, data access and storage services without the end user needing to know the physical location and / or configuration of the computers delivering a particular service. For example, the term "cloud" is used in this respect as a metaphor for the Internet (World Wide Web).For example, the cloud provides a computing infrastructure as a service (IaaS). The cloud computer can act as a virtual host for the operating system and / or data processing application used to execute the method of the invention. The cloud computer is for example the elastic compute cloud (EC2) provided by Amazon Web Services™. The computer is for example equipped with an interface for receiving or outputting data and / or performing analog-to-digital conversion. The data is for example data representative of physical properties and / or data generated from technical signals. The technical signals are for example generated by (technical) detection devices (such as for example devices for detecting marker devices) and / or (technical) analysis devices (such as for example devices for performing (medical) imaging methods), the technical signals being for example electrical or optical signals. The technical signals represent for example data received or output by the computer. The computer is preferably operatively coupled to a display device that allows the information output by the computer to be displayed, for example to a user. An example of a display device is a virtual reality device or an augmented reality device (also called virtual reality glasses or augmented reality glasses) that can be used as "goggles" for navigation. An example of such augmented reality glasses is Google Glass (a trademark of Google, Inc.). Augmented reality or virtual reality devices can be used both to input information into a computer by user interaction and to display information output by the computer. Another example of a display device is a standard computer monitor that includes, for example, a liquid crystal display operatively coupled to a computer to receive display control data from the computer to generate signals used to display image information content on the display device. A particular embodiment of such a computer monitor is a digital light box. An example of such a digital light box is Buzz®, a product of Brainlab AG.The monitor may also be the monitor of a portable, e.g. handheld, device such as a smart phone or a personal digital assistant or a digital media player.

[0063] The invention also comprises a program, which when executed on a computer causes the computer to perform one or more or all of the method steps described herein, and / or a program storage medium (particularly in non-transitory form) on which the program is stored, and / or a computer comprising said program storage medium, and / or a (physical, e.g. electrical, e.g. technologically generated) signal wave, e.g. a digital signal wave, carrying information representing a program, e.g. code means adapted to perform any or all of the method steps of the aforementioned program, e.g. as described herein.

[0064] Within the framework of the present invention, computer program elements may be embodied by hardware and / or software (which includes firmware, resident software, microcode, etc.). Within the framework of the present invention, computer program elements may take the form of a computer program product which may be embodied by a computer usable, e.g. computer readable data storage medium, such as a computer readable data storage medium comprising computer readable program instructions, e.g. a "code" or "computer program" embodied in said data storage medium for use on or in connection with an instruction execution system. Such a system may be a computer, which may be a data processing device comprising means for executing computer program elements and / or programs according to the present invention, e.g. a digital processor (Central Processing Unit or CPU) for executing the computer program elements, and optionally a volatile memory (e.g. Random Access Memory or RAM) for storing data used in and / or generated by the execution of the computer program elements. Within the framework of the present invention, a computer usable, e.g., computer readable, data storage medium can be any data storage medium that can contain, store, communicate, propagate, or transport a program for use on or in connection with an instruction execution system, apparatus, or device. A computer usable, e.g., computer readable, data storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a propagation medium, such as the Internet. A computer usable or computer readable data storage medium can be, for example, paper or other suitable medium on which a program is printed, since a program can be electronically captured, for example, by optically scanning the paper or other suitable medium, and then compiled, interpreted, or otherwise processed in a suitable manner.The data storage medium is preferably a non-volatile data storage medium. The computer program product and any software and / or hardware described herein form various means for carrying out the functions of the present invention in the exemplary embodiment. The computer and / or data processing device may include, for example, a guidance information device including a means for outputting guidance information. The guidance information may be output to the user, for example, visually by visual indication means (e.g., a monitor and / or a lamp), and / or acoustically by acoustic indication means (e.g., a speaker and / or a digital audio output device), and / or tactilely by tactile indication means (e.g., a vibration element or vibration elements integrated in the device). For the purposes of this document, a computer is, for example, a technical computer including technical, e.g. tangible components, e.g. mechanical and / or electronic components. Any device so referred to in this specification is a technical, e.g. tangible device.

[0065] BRIEF DESCRIPTION OF THE DRAWINGS The present invention will now be described with reference to the accompanying drawings, which provide a background description and depict certain embodiments of the invention, however the scope of the invention is not limited to the specific features disclosed in the context of the drawings, which are not to scale. [Brief description of the drawings]

[0066] [Figure 1] 1 is a schematic block diagram of a communication network having a data memory node according to a first embodiment; [Diagram 2] FIG. 2 is a schematic block diagram of a data encoder; [Diagram 3] FIG. 1 is a schematic block diagram of similarity calculation based on binary encoding. [Figure 4] FIG. 1 is a diagram of homomorphic encryption operations. [Diagram 5] FIG. 2 illustrates components of the network of FIG. 1 in more detail, according to one embodiment. [Figure 6]2 illustrates further components of the network of FIG. 1 in greater detail, according to one embodiment. [Figure 7] FIG. 11 is a block diagram of a communication network having a data memory node according to a second embodiment. [Figure 8] 1 is a flow chart of a method for anonymized fuzzy data matching. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0067] Description of the embodiments Reference is first made to the block diagram of Fig. 1, which illustrates in a schematic way a system DMS that may include a network of computing nodes. The nodes are arranged as computing devices. Each node is connected via a communication network CN. The communication network CN may be wired, wireless or both. The nodes may be arranged geographically separate from each other, but as will become clear below, they do not have to be. In one example, the communication network CN is arranged to implement the Internet protocol suite TCP / IP and related, without excluding other network types and protocols.

[0068] The network includes data custodians DC1, DC2. Only two are shown with the understanding that there may be more than two. Data custodians DC1, DC2 include data memory systems DM1, DM2, respectively. The data memories DM1, DM2 hold data. The data includes any type of data item envisaged herein, including structured or unstructured data. The data includes data items. The data items may be text, numeric, or alphanumeric. The data items may further or instead include video, audio, or a combination of any of the foregoing (multimedia). The data times may be organized into or form part of a data file.

[0069] The data memory systems DM1 and DM2 may be arranged as databases, although as mentioned above, this is not a requirement. In a database, data items may be arranged in records that include multiple fields, each representing a different attribute. Patient records in a medical database system such as a HIS (Hospital Information System) or a PACS (Picture Archiving and Communication System) are examples envisioned herein, as are packet traffic management systems for network switches or routers, malware databases, among many others. A "data item" as used herein may refer to the data in each field with attributes, or may relate to the respective data record as a whole.

[0070] The data custodian systems DC1, DC2 include respective data processing systems DP1, DP2 enabling data items in the respective data memories DM1, DM2 to be processed, manipulated, retrieved etc.

[0071] The data processing systems DP1, DP2 each include one or more processors (not shown). The processors include a central processing unit CPU (not shown) that communicates with a main or primary memory. During computational operations of the data processing systems DP1, DP2, data items and instructions can be loaded for processing data items held in the data memories DM1, DM2. The memories DM1, DM2 can be configured as secondary storage devices from which data items can be loaded into the main or primary memory as controlled by the respective CPUs of the data processing systems DP1, DP2.

[0072] The proposed network supports anonymous data matching, where data held by one data custodian, say DC1, may be queried against data held by the other data custodian DC2. Said data query involves matching data across two nodes DC1, DC2. Thus, the matching operation primarily envisaged herein is for membership, although other types of queries are not excluded herein. Although not excluded herein, exact matching, i.e., identity operations, are not primarily envisaged. In a preferred embodiment, what is envisaged herein is fuzzy matching based on a similarity measure. In other words, the result of the query or matching operation is an indication as to whether or not there is a match across data items held by the data custodians DC1, DC2, within a given similarity score or threshold q=q1 / q2. In this fraction, which is assumed to be irreducible, q1 is referred to herein as the (threshold) numerator, and q2 is referred to herein as the (threshold) denominator. The threshold quantifies the probability q% of a match. The sum q1+q2 may be referred to herein as the "threshold sum."

[0073] As will be discussed in more detail below, the query operations are anonymized so as not to leak data from one custodian to another, or indeed to any other third party. Thus, the operation of the network ensures that data privacy is protected based on cryptographic protocols. In addition to preserving data privacy, in preferred embodiments, the network uses probabilistic data matching that is robust to discrepancies in the data within a certain predefined similarity threshold q, usually described as a percentage.

[0074] Data matching based on similarity measures, or fuzzy matching, is useful not only for information retrieval applications, but also for storage optimization to determine the degree of duplicate data or sufficiently similar data, malicious file detection, malware detection, caching, distributed search, routing, forensic file systems, etc.

[0075] Although the embodiment of the network of nodes of FIG. 1 is configured as having two third parties, a linkage unit LU (or “linker”) and a decryption unit DU (or “decryptor”), this is not required and alternative embodiments are equally envisaged herein, in which no third parties are used or only a single third party is used. In the embodiment of FIG. 1, the fuzzy matching operation is performed in part by the linker LU in coordination with the decryption unit DU. One or both of the computational nodes linker LU and decryptor DU preferably include one or more processors capable of SIMD (“single instruction multiple data”) and / or parallel data processing, such as those having a multi-core design. GPUs may be used as processors in the linker LU and / or decryptor DU in embodiments. The data custodians DC1, DC2 appropriately prepare the data items to be matched by encoding, aggregation and encryption, as discussed more fully below. The data so prepared is then passed to the linker LU and decryptor DU. The linker LU and decryptor DU process the prepared data to generate the final result SR. The final result SR includes an indication of the match result. The final result is forwarded to one or both of the data custodians DC1, DC2. The result SR is provided in cleartext by the decryptor DU or in ciphertext that can be decrypted by one or both of the custodians DC1 and DC2.

[0076] The decryption unit DU and the linkage unit LU in some embodiments constitute additional nodes (third parties) in the network. However, this is not a requirement since the functions of the linkage unit LU and the decryption unit can be implemented by one of the custodians DC1, DC2 without a third party. In an embodiment, the system DMS supports data linkage operations across data held by the two data custodians DC1, DC2 to match and thus "link up" related data items that are sufficiently similar within the similarity measure used. The proposed system is configured to perform high-speed and low-latency matching in a privacy-preserving manner with reduced memory and / or network overhead.

[0077] More specifically, the fuzzy matching operation is based on a homomorphic encryption algorithm. Advantageously, as proposed herein, the fuzzy matching operation is executed by any of the data custodians, the linkage unit LU and / or the decryption unit DU in a SIMD setting. The fuzzy matching operation is preferably executed by processors of the data custodians DC1, DC2, the linker unit LU and / or the decryption unit DU configured for parallel computation. Multithreading with dynamic scheduling may also be employed by the processors. As will be explained more fully below, the data to be matched is represented in an efficient data structure that supports the execution of the fuzzy matching operation in a parallelized and / or SIMD manner. In a SIMD setting and related settings, the efficient data structure allows for processing multiple data items aggregated in the data structure with a single instruction. The data structure may be loaded as a whole into the main memory or at least into the virtualized memory of each processor in the respective computational entity (e.g., any one of the data custodians, the linker LU and / or the decryption unit DU). Furthermore, a single instruction that processes the data structure is loaded into the main memory to perform at least a part of the fuzzy matching operation. This allows data matching to be efficiently performed in main memory.

[0078] An efficient data structure is achieved by judicious aggregation of data items in the respective data custodians DC1, DC2, as will be explained more fully below. In one embodiment, a matrix representation is used. The fuzzy matching operation is managed, in part or in whole, by either the data custodian or the linker LU, and / or in part by one or more other third parties, such as the decryption unit DU. One application envisaged herein is matching patient records across a medical facility. Due to the similarity-based approach, the matching is robust to variations (within a threshold q) of data items related to the same patient. This allows for better memory management and can also support applications such as data analysis to better understand patient populations, etc. Although processing patient data is one embodiment envisaged herein, numerous other embodiments in network routing, forensic file systems, malware detection, etc., as described above, are all envisaged herein, as are all other applications that would benefit from anonymized fuzzy matching of data held in a distributed manner across nodes in a computer network.

[0079] In the proposed system, due to the representation of data items in the aggregated data structure, data items can be matched for similarity in an encrypted format in groups rather than individually and pairwise. Although the proposed framework contemplates matching only one data item from one data custodian with multiple data items of the other custodian, the advantages of the proposed system are most evident when multiple items from one data custodian are matched with multiple data items of the other custodian. Thus, the computation time is O(N 2Instead of matching two sets of data items from two data custodians in a pairwise fashion, which attracts a complexity of the order of O(N), where N is the number of data items, the proposed process proceeds to match multiple data items in a group-wise manner against multiple other data items held in the other data custodian at a time. The group-wise matching approach is O(N 2 ), but with a large hidden constant factor, namely the size of the aggregation, that contributes significantly to the reduction of the computational overhead. More specifically, if the quantity C is a constant block size over which data items are aggregated, then the new complexity of the algorithm is O(N 2 / C 2 ). In the best case, N=C, and only a single homomorphic operation is required, resulting in an exponential reduction in complexity. The proposed method reduces the number of homomorphic operations required compared to Cruz et al. (cited below), and therefore the number of resulting ciphertexts is also reduced by N 2 From (N / C) 2 Reduce to.

[0080] The operation of the system will now be described in more detail with continued reference to Figure 1. The proposed system preferably uses asymmetric homomorphic encryption schemes.

[0081] In operation, one or both of the data custodians DC1, DC2 wishing to match data requests a public encryption key from the decryption unit DU (1).

[0082] The data processors DP1, DP2 of the data custodians DC1, DC2 process and encode a plurality of data items into encodings. A binary encoding scheme is primarily envisaged herein, although others are possible. Each data processor DP1, DP2 then aggregates the respective encodings into a respective data structure, such as a matrix, as will be described more fully below. Each aggregation so formed is then encrypted by the data processors DP1, DP2 using a homomorphic encryption scheme. The two aggregates are each encrypted into a single ciphertext, which is then sent to (2) the linkage unit LU. Optionally, as will be described in more detail below in Figs. 5-6, the data custodians make available additional data (referred to herein as auxiliary data) that may facilitate the computation of the matching result.

[0083] The linkage unit LU then processes the two ciphertexts into a third single ciphertext by combining them. Since the encryption operations are homomorphic, this combination is preserved by the encryption applied. This third ciphertext is then sent to the decryption unit DU (3), where it is decrypted to obtain an intermediate plaintext structure.

[0084] A new output data structure, referred to herein as the output data structure, may then be transformed from said intermediate plaintext structure based on certain auxiliary information. The new output data structure SR (see Figs. 5-7) shows the final result, i.e., the matches between data items from the two groups held in the two data custodians DC1, DC2.

[0085] The anonymized computation of the similarity measure between the data items from the two groups is partly performed by the linker unit LU and in another, complementary part, by the decryption unit DU. More specifically, the combining of the two ciphertexts into a third ciphertext by the linker LU constitutes part of the similarity computation, while the remaining part of the computation of the similarity measure is performed in the decryption unit DU, and an output data structure SR representing the similarity match result is transformed from the auxiliary information and the intermediate plaintext structures.

[0086] Preferably, this output data structure SR only reveals whether there is a match between any two data items. However, the similarity level of the match and the identity of the matching data items are preferably not revealed to either party. The output data structure indicating the match may be sent specifically to one or both data custodians to inform them about the found match (4). Since the given custodian DC1 knows in what order the data items were originally aggregated by the given custodian DC1, it can easily determine from the output data structure the identity of the matching data items held by the given custodian DC1 by searching in the given custodian DC1, even if the given custodian DC1, or the linker LU or the decryption unit DU, have never actually accessed the data held by the other custodian DC2. Therefore, it is best to maintain the original order of the encodings in the aggregate data structure during the matching operation.

[0087] For example, the proposed system DMS can determine whether a data item is malicious or not, or if a match is found, whether a network router node should intercept the malicious traffic, or whether a patient in a medical facility DC1 has medical records in another facility DC2, etc.

[0088] Before describing the operation of the system in more detail in Figures 5 and 6, reference is first made to Figures 2-4 to describe specific computational primitives for encoding, similarity measurement, and encryption that can be used in the data matching system DMS proposed in this specification.

[0089] Referring now first to Figure 2, this shows a block diagram of an encoder EC that may be used by data custodians DC1, DC2 to encode their data items prior to encryption and matching. Preferably, each encoder EC uses the same encoding algorithm for both data custodians.

[0090] In general, the encoder EC transforms an input data item X="xxxx" to be represented as a combination E(X) of elements of an alphabet AL. In a binary implementation of the encoder EC, the alphabet AL contains only two elements AL=[a,b]. Without loss of generality, it can be assumed that a="1" and b="0".

[0091] Preferably, for efficiency of processing and better memory representation, the encoder is configured to map the input data X to a fixed length bit pattern of 0s and 1s. The length N is preset and may depend on the length and / or number of data items to be matched. In other words,

[0092]

number

[0093] It is. (Linear) bit pattern E(An example of a BP (X)=BP is shown diagrammatically at the bottom of FIG. 2, with the index j indicating the bit position. Higher dimensional input data, such as images, where the bit pattern is not linear but has a 2D matrix layout, are also envisaged. Preferably, the encoder EC has the property that its output, i.e. the bit pattern BP representing the encoding of the data item X, is uniformly distributed over the input X. One option, but not the only option, for achieving this is to use one or more hash functions operating on the input X. It is the hash value that is then mapped to the bit pattern. Suitable hash functions, or hash function types, include any combination of two or more hash functions selected from the SHA family of hashes (SHA256, SHA512), the MD family of hashes (MD5, MD6), or other random hash functions.

[0094] More specifically, as envisaged herein in an embodiment, a Bloom filter ("BF") or other probabilistic data structure (PDS) is used herein as the encoder EC. In Bloom filters and similar binary encoders, it may be necessary to pre-process the input X, for example if it is a string, before encoding. In this and similar embodiments, the input string X first undergoes n-gram analysis to be decomposed into a system of n-grams, which are a set of substrings of length less than the length of the original string X. The concept of BF is extended to be applicable to numerical data as well. Thus, X can be a string (such as a patient's name), a number, or an alphanumeric character, as previously described. The encoding operation in both custodians DC1, DC2 is preferably based on the same probabilistic data structure with the same hash function and the same encoding length of the encoding output.

[0095] In an embodiment, bigrams are used with n=2, and data item X is decomposed into a sequence of substrings, each of length 2. Each of these bigrams is then hashed by a different hash function, the number of which may be 2, 3 or more and is predefined.

[0096] In the BF embodiment, a modulus calculation (mod N) is then performed on each hash value of each n-gram to map it to one or more bit positions. In the modulo N operation, a modulus value of N is predefined. Preferably, N is equal to a fixed length of the bit pattern BP. Each hash value corresponds to one or more bit positions. The encoder EC then records one bit for each of the calculated bit positions, deriving with a particular bit pattern of X, thus performing the encoding. Each n-gram generally contributes to two or more bit positions, depending on the number of hash functions used. Preferably, two or more hash functions are used. Each bit pattern generated by the encoder can be understood as a digest of the original data item X. As will be explained more fully in FIG. 3 below, it is the Bloom filter BF in particular, or more generally, the binary encoder, that allows for efficient processing of data for similarity. It will be understood that the above encoding operation is repeated for each data item that is matched across the two data custodians DC1, DC2, with each data custodian DC1, DC2 performing the encoding on its respective data item.

[0097] The respective bit pattern EC(X)=BP thus derived for a data item X allows one to define an appropriate similarity measure: intuitively, the more similar the patterns E(X'), E(X) are, the more similar the respective underlying data item inputs X', X are.

[0098] While a binary encoder is preferred herein, this is not a requirement herein as long as all inputs X are mapped to the same alphabet. Preferably, the length of the encoder output is constant for all inputs X. For ease of comparison, it is preferred that both custodians use the same alphabet and a constant length N. Again, the mapping to a constant length code can be ensured by using a hash function, although other techniques are envisioned.

[0099] It should be noted that the above encoding is optional. In some cases, the data to be matched is already native in a format that can be processed herein, or such data can be processed differently than the encoding described above. For example, image data may be available in a bit format, such as a bitmap format, which is another type of binary data contemplated herein. However, non-image data, such as text or numbers or alphanumeric combinations, may need to be first encoded as described above with a BF or other binary encoder. The terms "filter" and "encoder" are used interchangeably herein.

[0100] The way in which the similarity measure of the encoded data E(X) is calculated will now be described with reference to FIG.

[0101] As mentioned above, the task of calculating the similarity between the encoded data E(X) is preferably, but not necessarily, divided between the linkage unit LU and the decryption unit DU. In an embodiment, it is assumed that at least two computational entities, namely the linkage unit LU and the decryption unit DU, each calculate a certain part of the similarity measure, which can then be converted into a complete similarity measure by a computational entity of the system DMS, such as the decryptor DU or the linker LU (or one of the custodians or even another entity). The similarity result according to the complete similarity measure makes it possible to check whether there is a match across the data items held by the two custodians.

[0102] 3 shows two exemplary encodings E(X1), E(X2) of two data items X1 and X2. X1 is a data item held by data custodian DC1, and X2 is a data item held by custodian DC2. The two bit patterns E(X1), E(X2) have the same length N as mentioned above, so they can be aligned and compared. Specifically, by aligning the binary encodings E(X1), E(X2), it is possible to find the common bit position j where there is an "overlap", i.e., the two encodings have the same common bit position j, E(X1), E(X2), respectively. j =E( X 2) j =1 records 1 bit.

[0103] Bit patterns can be viewed as discrete sets, and common bit positions represent the intersection of those two sets. MSH Cruz et al., "Secure similarity joins using fully homomorphic encryption," iiWAS'17: Proceedings of the 19th International Conference on Information Integration and webbased Applications&Services, December 201 7 As reported elsewhere in, pp. 224-233, several similarity measures have been defined in the past to compare how similar discrete sets are. One such similarity measure includes the Tversky index. Examples of other similarity measures contemplated herein include the Jaccard index or the Dice coefficient.

[0104] In general, similarity measures for discrete sets are functions of the set-theoretic union and set-theoretic intersection of the two sets being compared. More specifically, it has been observed that the Tversky index as well as similar indices can be written as functions of the union count UC and the intersection count IC. The counts represent the size of the intersection or union set, in other words, the number of elements of the two sets. For present purposes, the elements are all the one-bit positions of the two encodings for the union count and the common one-bit positions for the intersection count.

[0105] Indeed, a large family of similarity measures for discrete sets, referred to herein as Jaccard-type measures, can be understood as intersections of unions.

[0106]

number

[0107] The Tversky index is an example of (1) and is assumed herein in the embodiments. Based on the Tversky index, the Tversky threshold index measure "TI(·)" can be derived. In addition to the Tversky threshold index measure, other types of similarity measures can also be written as linear combinations of the union set count UC and intersection count IC with similarity thresholds q, q. For example, Cruz et al., cited above, report the following findings for binary data:

[0108]

number

[0109] where q1, q2 are similarity thresholds q. The Tversky index scale allows for a binary form of similarity matching. It is noted that no information on the level of similarity is disclosed. More specifically, the Tversky index scale can be positive or negative. A non-negative TI scale indicates a match with a similarity of q% or more, and a negative scale indicates a mismatch since the similarity is less than q%. To ensure even better privacy protection, it is particularly fuzzy data matching based on the TI scale that is envisaged in the embodiments herein. However, matching based on other, more general scales according to (1) is not excluded herein. As can be understood, in the example of scale (2), the above conversion from the partial scale to the complete scale is a linear combination with threshold data (threshold sum and numerator in this case) as further auxiliary data.

[0110] As shown in FIG. 3, in an embodiment, the proposed similarity matching system DSM is configured such that the linkage unit LU together with the decryption unit DU can process the union set counts and intersection set counts according to (1) into a complete similarity measure to obtain similarity results for two groups of data items, one group from data custodian DC1 and the other group from data custodian DC2.

[0111] As described in more detail below, the computation of the Tversky measures, including intersection and union counts, can be efficiently implemented by representing the encoded data items E(X) in a matrix structure. This allows the use of SIMD methods described above and / or parallelizing the execution to efficiently compute the set-theoretic counts required for the similarity measures, i.e., intersections and / or unions. Pairwise comparisons are no longer necessary, as computations can be done group-wise at a time for multiple data items to be matched.

[0112] The proposed system DMS is configured to perform the above computation of union and / or intersection counts in an anonymous manner. That is, the data items themselves are not only coded but also encrypted. A homomorphic encryptor ENC is preferably used together with a packing function, typically based on the Learning with Errors (LWE) problem. The encoder EC does not result in encryption, even when using Bloom filters. It has been reported that Bloom filters can be brute-forced to reveal the underlying data X by frequency and cryptanalysis. The proposed system further comprises an encryptor ENC, which preferably performs homomorphic encryption.

[0113] Homomorphic encryption ("HE") is now described in more detail in FIG. 4. Not all encryption schemes are homomorphic. Those that are homomorphic and are desirable for use in the proposed system include the Brakerski-Gentry-Vaikuntanathan (BGV) scheme, the Gentry-Saha i-Examples of homomorphic encryption include the GSW (German Waters) scheme, and the Brakerski / Fan-Vercauteren (BFV) scheme, but any homomorphic encryption scheme that supports ciphertext packing (particularly matrix multiplication) can be used. As shown in the communication diagram of Figure 4, a data item m is given in plaintext in the plaintext domain PD. The data item m can be subjected to an operation f to obtain a plaintext result f(m) (4) in the plaintext domain PD. The operation f can be, for example, an arithmetic operation such as multiplication or addition or a combination thereof to form a linear combination, in particular an inner product operation, or matrix multiplication, etc. "Homomorphic" encryption means that the encryption operation enc(·) preserves or supports the plaintext operation f. That is, the same result f(m) can be achieved by first (1) encrypting m to obtain the ciphertext enc(m) in the ciphertext domain CD, (2) applying the corresponding homomorphic operation F to obtain the result F(enc(f(m))) in the ciphertext domain, and then (3) decrypting the result to recover the previous plaintext result f(m). The exact nature and functionality of the corresponding ciphertext domain operation F depends on i) the operation f and ii) the particular homomorphic encryption algorithm. For example, as will be explained in more detail, the plaintext operation preferably comprises a matrix multiplication, and the encryption performed by the encryptor ENC is preferably selected to support this operation f. The encryption schemes BGV, GSW, BFV mentioned above, and their variants (in particular those based on the learning with errors (LWE) problem), are suitable to support matrix multiplication, linear combination, etc., resulting in a suitable corresponding ciphertext domain operator F.

[0114] The encryptor ENC proposed herein is not only configured for homomorphic encryption, but also configured to perform a packing operation to encrypt an entire data structure, including a collection of multiple data items, such as a matrix or a vector, into a single ciphertext. In an embodiment, this can be done by the encryptor ENC or a pre-processing entity, which first maps the data structure into a polynomial. More specifically, the data items constituting the data structure are embedded in the coefficients of said polynomial. The polynomial is then encrypted by the encryptor ENC with an appropriate HE scheme. A homomorphic operation F is then performed on the encrypted polynomial. Other homomorphic encryption schemes with packing are also envisaged herein in an embodiment. Packing techniques exist for both matrices and vectors.

[0115] In the proposed system, the encodings at each data custodian are aggregated into a suitable data structure by the respective aggregator AG, AG' before packing and encryption. More specifically, in the embodiment envisaged herein, the encodings, i.e., bit patterns, are arranged as rows or columns of a matrix structure to perform the aggregation. Thus, one matrix Two matrices M1, M2 are obtained: M1 and one matrix M2 by another data custodian DC2. The above-mentioned packed encryption scheme can map the matrices into the coefficients of respective polynomials, and then each polynomial can be encrypted into a respective single ciphertext by the HE algorithm. In other words, the entire collection of data items, i.e., matrix M1 or M2, respectively, becomes a single ciphertext {M1}, {M2}. The single matrix ciphertext facilitates the computation of an efficient SIMD-style similarity measure, as described above in FIG. 3.

[0116] Preferably, the two matrices M1, M2 into which the encodings from the two custodians are respectively aggregated are transposes of each other. In other words, the encodings of the data items in one custodian DC1 are aggregated, for example, as one row of the matrix M1, while the encodings of the other custodian are aggregated as columns of the other matrix M2, or vice versa. It is not necessary that the matrices are square. The dimensions of the matrices may depend on the length of the respective encodings and the number of data items in the two groups to be matched.

[0117] The matrix multiplication involves a row and column dot product. For encoding into binary bit patterns, the dot product directly results in intersection counts for each element of the product matrix M1*M2 by multiplying relatively transposed matrices containing the respective data items from the custodians DC1, DC2. The matrix multiplication can be efficiently implemented as a parallelized operation in a SIMD-enabled architecture. Thus, the intersection counts can be obtained group-by-group at once, rather than sequentially pair-by-pair, making the system DMS more responsive. However, it will be appreciated that the requirement for a transposition relationship between M1 and M2 is at least partially determined by the nature of the encoding and / or the functional representation (1) of the similarity measure. Such a transposition relationship, and / or different matrix operations, may not be required by functional representations other than (1), and / or encodings other than binary. For homomorphic packing and encryption schemes, the above-mentioned intersection count operation by matrix multiplication can be performed in the ciphertext domain to compute {M1*M2}, which is a single ciphertext of the product matrix M1*M2 that records the intersection counts.

[0118] The operation of the fuzzy data matching system DMS will now be described in more detail with reference to the block diagrams of Figures 5 and 6. Figures 5 and 6 are according to an embodiment in which the encoding of the data items is collected in matrices M1, M2. The block diagram of Figure 5 mainly concerns the operation of the linkage unit LU, while Figure 6 shows in detail the operation of the decryption unit. However, as will be explained in more detail below, the two third parties are not necessarily required. The numerical components of the matrices in Figures 5 and 6 (and Figure 7) are purely for illustrative purposes. In the example of Figures 5 to 7 below, q1=4, q2=5, so the threshold q=4 / 5, and the threshold sum q1+q2=9. In the matrices, "RC" denotes the row count, "CC" denotes the column count, and "Cti", i=1,2, denotes an exemplary count. "Ai" denotes the data items of data custodian DC1, and "Bi" denotes the data items of data custodian DC2. "Biti" is a general reference to the bit entries of the encoding.

[0119] 5, the decryption unit DU generates a homomorphic encryption key pair and shares the public key with the data custodians DC1, DC2. The private key remains with the decryption unit DU.

[0120] In some, but not all, embodiments, the data items to be matched are pre-processed to facilitate the encoding operation by the encoder EC. For example, each data custodian DC1, DC2 may split and match the data items it holds (such as patient demographics) into n-grams, such as bigrams (n=2), and then hash by the encoder EC into a record Bloom filter (RBF) for each data item, for example, for each patient record. In other words, the entire data record for each patient may be hashed in this way. However, again, other data items, such as individual attributes, may be hashed in this way instead. Although the reference to RBFs in this specification serves as an example rather than a limitation, such RBFs are still envisioned in the embodiments. Record-by-record encoding is not limited to the medical field.

[0121] The encodings of such RBFs are then aggregated by aggregators AG, AG′ into respective RBF matrices M1, M2. This aggregation operation can be conceptualized by “stacking” the encoded data items RBFs “on top of” each other to generate respective RBF matrices M1, M2 at each data custodian DC1, DC2.

[0122] Next, the encryptor ENC generates two ciphertexts {M1}, {M2 T} is generated, while {M1} is encrypted to {M2 T} is generated by transposing and encrypting. The matrix packing technique is used to pack each input matrix M1, M2 T Preferably, one of the facilities DC2 first obtains a single ciphertext for the matrix M2 of the other facility DC1 in a transposed form. T It will be appreciated that the active transposition step is not necessarily required since it is possible to generate T It is assumed that M1, M2 are generated such that the data items aggregated in them are the transposes of each other. Whether this is achieved by active transposition or natively is irrelevant here. To keep the notation simple, let us understand M1, M2 to be the transposes of each other and write " T " Drop it.

[0123] Auxiliary data XD in the form of two additional vectors CV1, CV2 (referred to herein as count vectors) are generated at the respective data custodians DC1, DC2 by the respective count processors CP. The count processors CP may be efficiently implemented by a series of dot product operations with correspondingly transposed unit vectors. As a result, each count vector contains in plaintext the sum of one bit from each row (in the case of CV1) and each column (in the case of CV2) of the respective RBF matrix M1, M2, and thus the two vectors CV1, CV2 inherit their transposition relationship from the respective matrices M1, M2. T " notation is also dropped, and it is understood that CV1 and CV2 are also transposes of each other relative to each other.

[0124] The converter CONV converts the count vectors CV1, CV2 by scalar multiplication with the numerator q1 of the Tversky exponent threshold q. The converted count vector CV1 is then + , CV2 + are encrypted by the encryptor ENC into one separate ciphertext each, preferably using a vector packing technique. V The notation "+" is used to denote the operation of . It is understood that a transformer operation may be used multiple times herein, which may or may not be the same computational entity.

[0125] All the above four ciphertexts send the matrices {M1}, {M2} and the count vectors {CV1}, {CV2} of the two custodians DC1, DC2 to the linker LU, where they are received at the input interface IN.

[0126] The linker LU reweights the counts {CV1}, {CV2} by multiplication with the plaintext scalar q1 or the ciphertext scalar {q1} using the transformer CONV. Preferably, the plaintext q1 is used since it can be assumed that homomorphic encryption supports plaintext multiplication in the ciphertext domain. The linker LU selects the corresponding row counts {CV1} and encrypted RBF matrix {M1} from one facility and the column counts {CV2} and the relatively transposed and encrypted RBF matrix {M2} from the other facility. The two RBF ciphertexts are multiplied by the combiner Π to produce the multiplied matrix ciphertext {M1*M2}. This matrix product ciphertext is then transformed by multiplication with the scalar threshold sum q1+q2 to produce the transformed product {(M1*M2) +} is obtained. The threshold q and the sum q1+q2 may be publicly accessible to perform this transformation, or may be provided by the custodians DC1, DC2 as part of the auxiliary data XD. As a variant, this transformation by multiplication of the threshold sum q1+q2 may instead be performed in one of the custodians DC1, DC2, rather than in the linker LU. For example, one of the plaintext input matrices M1, M2 may be multiplied by the threshold sum q1+q2 in the data custodians DC1, DC2.

[0127] The transformed matrix product ciphertext {(M1*M2) +} is then sent as auxiliary data to the decryptor unit DU along with the selected row and column count vector ciphertexts {CV1}, {CV2}.

[0128] Turning now to the operation of the decryption unit DU in FIG. 6, the decryption unit DU derives the three received ciphertexts {(M1*M2) +}, {CV1}, and {CV2}. The decryption unit DU operates in the plaintext domain, in contrast to the linker LU, which operated in the ciphertext domain.

[0129] By unrolling the row count vector CV1, a row count matrix CM1 (with the same dimensions as the multiplied matrix M1*M2) is generated with each element of the row matrix as a row. Similarly, a column count vector CV2 is used to generate a column matrix CM2 with each element as a column in the column matrix. Again, the dimensions of CM2 are the same as those of M1*M2. This dimensional information is available for the plaintext data. Even though DU operates in plaintext, the correspondence of data items versus rows / columns is stored in the data custodians DC1, DC2, DC3, DC4, DC5, DC6, DC7, DC8, DC9, DC10, DC11, DC12, DC13, DC14, DC15, DC16, DC17, DC18, DC19, DC20, DC210, DC22, DC23, DC24, DC25, DC26, DC27, DC28, DC29, DC30, DC31, DC32, DC33, DC34, DC35, DC36, DC37, DC38, DC39, DC40, DC41, DC42, DC43, DC44, DC45, DC46, DC47, DC48, DC49, DC50, DC51, DC52, DC53, DC54, DC55, DC56, DC57, DC58, DC59, DC61, DC62, DC63, DC64, DC65, DC65, DC66, DC67, DC68, DC69, DC70, DC71, DC72, DC73, DC74, DC75, DC75, DC76, DC77, DC78, DC79, DC79, DC79, DC79, DC80, DC81, DC82, DC83, DC84, DC85, DC85, DC96, DC97, DC98, DC99, DC1 C It will be understood that the data will not actually be revealed since it is only known to the two parties.

[0130] The converter CONV' of the decryption unit DU then subtracts the generated row and column count matrices CM1, CM2 from the multiplied matrix M1*M2 to generate a plaintext result matrix SR. The result matrix SR, or relevant elements within the result matrix, may be forwarded to one or both of the custodians DC1, DC2 for evaluation.

[0131] The result matrix SR can record similarity measures within a threshold q. In other words, an element in the result matrix can indicate a match score higher than the threshold q%. For example, q=80%. In a Bloom filter, or other PDS encoding, a non-negative (≧0) element represents a match score higher than the threshold q%. However, a negative element (<0) indicates a similarity score below q%, which is a mismatch.

[0132] As is clear from the description of Figures 5 and 6, the proposed system is not necessarily tied to aggregation into 2D matrices. Other similar data structures that allow efficient processing, especially parallelized processing, are also envisioned in the embodiments herein, such as higher than two-dimensional matrices, "tensors," etc. In other embodiments, data structures that are not matrix-like at all may also be envisioned.

[0133] Although the above-described embodiment is based on a data custodian supported by two third parties, namely a linkage unit LU and a decryption unit DU, this is not necessarily the case in all embodiments. For example, in one alternative embodiment, the functions of the linkage unit LU and the decryption unit DU are integrated into a single combined processing entity LU', so that only one third party is required. However, the proposed scheme can also be implemented with advantage without any third parties at all, with the functions of the decryption unit and the linkage unit being fully performed by one of the data custodians.

[0134] More specifically, in an embodiment with a single third party LU' (not shown), the two custodians DC1, DC2 transfer four ciphertexts to the linker LU', which are transformed by multiplication with two encrypted input matrices {M1}, {M2}, which represent common set counts that are transposed relative to one another, and q1, to produce two row and column count matrices {CM1}, {CM2}, which are also transposed relative to one another, as shown in FIG. +}{CM2 + The combiner Π and converter CONV′ of the linker LU operate in the ciphertext domain to combine {M1} and {M2} into a matrix product multiplied by the threshold sum q1+q2, resulting in {(M1*M2) +}. The converter then generates {(M1*M2) +} to {CM1 +}, {CM2 +} to generate a match result matrix {SR}, which can be forwarded to one or both of custodians DC1, DC2 for decoding and evaluation.

[0135] In an embodiment without a third party (not shown), the calculation is + This is performed peer-to-peer by one of the custodians, DC2, after receiving ciphertext data containing count {CM2+} and {M2}. The receiving peer DC2 then performs the operations as described in the previous paragraph for the linker LU. A copy of the result matrix {SR} is then returned to the custodian DC1, where it is decrypted and evaluated. Note that the custodian DC2 does not have to decrypt the result {SR}, since we are assuming here that DC1 generated the encryption key and is the only party capable of decrypting {SR}.

[0136] In the embodiments, particularly in FIG. 5 and FIG. 6, when implemented as shown with two third parties, a single third party, or no third party, the intersection count IC and the union count UC are separately encoded in vectors as auxiliary count data XD=CV1, CV2 generated by the respective counter modules CP. A more unified system architecture is also envisioned when all relevant operations can be performed by matrix operations without such count auxiliary data XD. Specifically, it is no longer necessary to explicitly provide the count data CV1, CV2 as described above in the embodiments of FIG. 5, FIG. 6. One such embodiment for this unified system architecture is shown in FIG. 7. In the embodiment of FIG. 7, only the dimensions (e.g., n×m and m×l) of the two matrices M1, M2 are provided or the dimensions can be made public, without the count vectors CV1, CV2.

[0137] In this embodiment, as previously described in Figures 5 and 6, the encoding is aggregated and the two matrices M1, M2, one of which is the transpose of the other M2, are transferred in ciphertext to a composite entity LU' that integrates some or all of the functions of the linker LU and the decryptor DU. However, the embodiment of Figure 7 may also be implemented in the form of a two third party, with separate entities with which the linker LU and the decryptor DU communicate, or indeed in a zero third party embodiment.

[0138] The following operations are performed in the ciphertext domain by the composite entity LU': The two separate ciphertexts {M1} and {M2} are combined by the combiner Π into a single third ciphertext {M1*M2} that represents the matrix product M1*M2. Hence the intersection count. In this embodiment, in addition to the two matrix ciphertexts {M1}, {M2}, only the dimensions of the plaintext matrices M1, M2 need to be included as auxiliary information.

[0139] As mentioned before, the numerator and denominator of the threshold may also be transmitted to the linker LU by the data custodian DC1 or DC2 in plaintext or ciphertext. Alternatively, the numerator and denominator of the threshold may be made public and requested by the linker composite LU'. With this threshold information, the similarity match can be performed entirely based on matrix multiplication as shown in FIG. 7. In particular, the converter CONV multiplies the matrix multiplied ciphertext {M1*M2} by a threshold sum, i.e., the sum of the numerator q1 and denominator q2 of the threshold q, in the ciphertext domain. The result is the transformed matrix product {(M1*M2) +}.

[0140] 5-7, the operation of the converter CONV is generally a function of the similarity measure one wishes to implement. In the embodiment envisaged herein, the converter CONV implements a similarity according to equations (1) and (2). The operation of the converter CONV is, in an embodiment, one of i) reweighting based on a threshold q, and ii) forming a linear combination by subtraction, although this converter operation may be different for similarity measures other than of the type of equation (2).

[0141] The two auxiliary matrices {MX1}, {MX2} are generated by the linker LU' based on the dimensions of the input matrices M1, M2 contained in or published in the auxiliary information received from one of the data custodians DC1, DC2. The auxiliary matrices {XM1}, {XM2} are generated as respective identity matrices and then multiplied by a threshold numerator q1. {MX1}, {MX2} are transposes of each other, corresponding to the input matrices M1, M2, which are transposes of each other.

[0142] Then, the ciphertext matrices {M1}, {M2} are multiplied by a combiner Π with auxiliary matrices {XM1}, {XM2}, which are transposes, respectively, to obtain two additional matrices {M1*MX1}, {M2*MX2}. Note that, as above, here the count data CV1, CV2 do not need to be provided by the custodian, but instead, said count data are calculated "internally" by the composite unit LU'. The internal calculation of the counts is performed by multiplications {M1*MX1}, {M2*MX2}. In other words, the merged set count matrix previously generated by the decryptor DU in Fig. 5, Fig. 6 is calculated internally in the composite unit LU' as the ciphertext.

[0143] The converter CONV converts the multiplied matrices M1 and M2 into two additional matrices {M1*MX1}, {M2*MX2} and the expanded ciphertext {(M1*M2) +} to construct a ciphertext result matrix {SR}. The result matrix {SR} contains the similarity results in the ciphertext. Specifically, according to one embodiment, according to formula (2) of the structure of the similarity measure TI(·), two additional matrices are subtracted by the converter CONV from {M1*M2} to obtain a result matrix of the ciphertext {SR}. The result matrix contains as components data item match results with rows and columns corresponding to the data times of one data custodian DC1 and the other data custodian DC2, respectively. The ciphertext {SR} result matrix can then be transferred to the custodian DC1 or DC2 for decryption, or to any other party holding a private encryption key associated with the public key used for encryption.

[0144] From the above, it will be appreciated that the matrix multiplication operations of Figures 5-7, in which one input matrix M1 is the transpose of the other matrix M2, constitute an efficient instruction for computing the partial similarity measures, in particular the intersection count data, as observed above in relation to Figure 3 and equations (1), (2). The matrix operation M1*M2 thus represents the intersection count IC. What the applicant has discovered herein is that the "mechanics" of matrix multiplication can be exploited to perform an efficient implementation of the computation, in particular the intersection count IC. The auxiliary vectors or matrices used above constitute a representation of the union count UC. The converter CONV processes the union count and the intersection count by subtraction, as in equation (2) above. For example, the row / column counts used in the embodiments of Figures 5, 6 and output by the counter CP may be implemented by dot product multiplication with a vector whose components are all ones.

[0145] However, it should be noted that for any of the above-described embodiments of Figures 5 to 7, the exact configuration of the converter operation may be dictated by the functional expression of the similarity measure one wishes to calculate, so that the exact operational characteristics of the converter CONV may not necessarily be subtractive, which may be sufficiently different from the embodiment according to equation (2).

[0146] Although Fig. 7 is described with respect to a single third party, the composite entity LU', this embodiment can also be implemented with two third parties, i.e., the linker LU and the decryptor DU, as in Figs. 5, 6, or without a third party peer-to-peer with one of the custodians DC2. In an embodiment without a third party, the encryption key can be generated by one of the custodians, e.g. DC1, where the other custodian, e.g. DC2, performs the similarity calculation. In an embodiment with two third parties, the decryptor DU can generate the key and make the result SR available in the clear. In both embodiments, no count data is required, only matrix dimension information is required.

[0147] The systems of Figs. 1, 5 to 7 use up to four ciphertexts instead of 2N ciphertexts, thus making it possible to achieve a reduction in communication overhead, and 2 Produces a single ciphertext {SR} as a result instead of M ciphertext results. Matrix multiplication to obtain M*M is computationally more efficient than pairwise multiplication of the encrypted encodings (e.g., Bloom filters).

[0148] Preferably, a computation entity that performs the matrix multiplication (e.g., a linker LU, LU' or a custodian D C The main memory of one of DC1, DC2) is configured such that the input matrices, e.g., M1, M2, may each be loaded in their entirety into the main memory. Alternatively, matrices M1, M2 may be split into smaller matrices that are each multiplied together, so that each of the smaller matrices may be loaded in their entirety into the main memory. This allows for faster comparisons than pairwise comparisons.

[0149] In the above embodiment, the auxiliary data counting vectors or matrices may be packed (if non-scalar) and encrypted for transfer to a third party or other data custodian. The auxiliary data may include matrix dimension information or it may be transmitted in the clear.

[0150] The proposed scheme described herein assumes that the linkage units LU, LU' and decryption units DU are protocol-following, honest but curious, and do not collude with each other.

[0151] Reference is now made to Figure 8, which shows a flow chart of a method for performing anonymous data matching between data memories in a network. The data matching is fuzzy in that only similarities within a threshold q are sought. Identical matches are not required here, but can still be performed by choosing, for example, q=1.

[0152] The top part of FIG. 8 shows two chains, the left chain representing respective steps performed by one of the custodians DC1 with corresponding steps performed by the other data custodian DC2 shown as the right chain.

[0153] In steps S810, S810', the data items to be matched are accessed in each custodian. Preferably, each data custodian DC1, DC2 accesses a plurality of its data items to be matched in steps S810 and S810' to define two groups of data items, one group held by data custodian DC1 and the other group held by data custodian DC2.

[0154] Step S820, S820 ’ The data items are then preferably encoded in binary format to obtain two groups of encodings for each of the two sites DC1, DC2. Steps S810, S810' are optional if the data is already in a suitable format, such as binary format.

[0155] In steps S830, S830', the encodings of the data items are respectively aggregated into respective aggregate data structures, such as respective matrices. The two matrices are preferably transposed with respect to each other. In other words, in one data custodian the encodings are arranged in rows, while in the other custodian the encodings of the data items are arranged in columns, or vice versa. This step is preferably coordinated, with the parties agreeing on what is aggregated in rows and what is aggregated in columns. A communication protocol may be set up to implement this coordination.

[0156] In steps S840, S840', each matrix is ​​packed and encrypted to derive a single ciphertext of the two matrices at the two data custodians DC1 and DC2, respectively. Since the matrices are transposes relative to each other, one of the matrices may be first inverted and then packed, and the other may be packed without inversion. The two matrix ciphertexts are then transferred to a third party, although this may not be the case in all embodiments, since further processing steps may be performed either at the custodian or peer-to-peer.

[0157] In step S850, the matrix ciphertext is processed into a third single ciphertext. The single third ciphertext represents a combination of the two aggregates. In one embodiment, the single ciphertext represents the ciphertext of the multiplication of the two input matrices generated in steps S830, S840. Step S850 represents the computation of an intersection count. Multiplication is supported in the ciphertext domain by homomorphic encryption (HE) applied in steps S840, S840'. HE supports matrix multiplication and / or inner products.

[0158] In step S860, the third ciphertext of the matrix multiplication is decrypted to reveal a data structure representing the partial similarity measures of the data items that match. In particular, the partial similarity measure may be the intersection count or a quantity proportional thereto.

[0159] In optional step S870, the decrypted matrix multiplication intersection counts as revealed in step S860 are converted into a plaintext representation of the complete similarity measure for the part or each data item of the other custodian based on auxiliary information. The auxiliary information may include, among other things, the union set counts of bits in the encoding. In an embodiment, the similarity measure (or score) is binary and simply represents whether a match has been recorded for a given pair of data items. In an embodiment, the transformation S870 may be performed in the ciphertext domain, and the decryption step S860 is applied after the transformation S870, which then results in a complete similarity measure of the plaintext rather than a plaintext partial similarity measure. In other words, in an embodiment, the method may operate entirely in the ciphertext domain.

[0160] The method may be performed in whole or in part by a single or two third parties that receive the encrypted input and auxiliary data from the custodians, or the method may be performed peer-to-peer by one of the custodians.

[0161] Preferably, the similarity scores are made available in a matrix structure inherited from the previous step, such that element (i,j) represents the similarity result between data item i from one data custodian and data item j from the other data custodian. However, the representation of the similarity match result in such a matrix structure is not mandatory and may be provided in other forms.

[0162] In that case, the similarity match results may be made available to one or both of the data custodians, or may be made available to other parties in accordance with the protocol. As will be understood, references herein to "match results" are general and include results that indicate that two data items do not match.

[0163] It should be noted that the embodiments of the present invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims, and other embodiments are described with reference to apparatus type claims. However, those skilled in the art will understand from the above description that, unless otherwise stated, any combination of features belonging to one type of subject matter, as well as any combination between features related to different subject matters, is considered to be disclosed with this application. However, all features can be combined to produce synergistic effects that are more than the simple sum of the features.

[0164] While the invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered as exemplary or illustrative and restrictive. The invention is not limited to the described embodiments. Other variations of the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the dependent claims.

[0165] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims, whether they are numeric, alphanumeric or a combination of one or more letters, shall not be interpreted as limiting the scope.

Claims

1. 1. A computer-implemented cryptographic method for facilitating data similarity queries across a network of data memories, comprising: a) receiving (S810) from a first memory (DM1) in the network a first ciphertext {(M1)} for a first aggregation (M1) of a plurality of first encodings of a first group of a plurality of data items; b) receiving (S810′) from a second memory (DM2) in the network a second ciphertext ({M2}) for a second aggregation of the second encodings of the second group of data items; c) processing the first and second ciphertexts into a third ciphertext ({M1*M2}) decipherable to a plaintext output (M1*M2), the plaintext output indicating or convertible to a similarity measure between data items from the two groups (S850); and Including, the first and second aggregations are pre-obtained by forming respective at least two-dimensional (2D) matrices from the first encoding and the second encoding; The method of claim 1, wherein the process c) comprises a matrix multiplication of the at least two matrices by a processor capable of single instruction multiple data SIMD processing.

2. The method of claim 1 , wherein the first encoding and / or the second encoding is a binary encoding.

3. The method according to claim 1 or 2, wherein the first encoding and / or the second encoding is by representation in at least one probabilistic data structure PDS.

4. The method according to any one of claims 1 to 3, wherein the first and second ciphertexts are previously obtained by respective homomorphic encryption operations.

5. The method of any one of claims 1 to 4, further comprising decrypting (S860) the third ciphertext into a plaintext output.

6. 6. The method of claim 1, wherein the plaintext output represents a partial similarity measure that can be converted into the similarity measure, and / or the similarity measure or the partial similarity measure is based on any one of the following measures: Dice coefficient measure, Tversky coefficient measure, Jaccard coefficient measure.

7. The method of any one of claims 1 to 6, comprising converting (S870) said plaintext output into said similarity measure.

8. 1. A method for facilitating data similarity queries across a network of data memories, comprising: a) forming (S830, S830') i) a first aggregation (M1) from a plurality of first encodings of a first group of a plurality of data items in a first memory (DM1) and ii) a second aggregation (M2) from a plurality of second encodings of a second group of a plurality of data items in a second memory (DM2); b) homomorphically encrypting the two aggregates into two ciphertexts ({M1}, {M2}) (S840, S840'); Including, the first and second aggregations are obtained by forming respective at least two-dimensional (2D) matrices from the first encoding and the second encoding; The two ciphertexts are processed into a third ciphertext ({M1*M2}), the processing comprising a matrix multiplication of the at least two matrices by a processor capable of single instruction multiple data SIMD processing.

9. 1. A method for facilitating data similarity queries across a network of data memories, comprising: decrypting a third ciphertext ({M1*M2}), previously obtained by processing the first ({M1}) and second ({M2}) ciphertexts, into a plaintext output that can be converted into a similarity measure of data items across groups; said first ciphertext ({M1}) having been previously obtained by homomorphic encryption of a first aggregation (M1) of a plurality of first encodings of a plurality of data items of a first group in a first memory (DM1); Decrypting (S860), wherein the second ciphertext ({M2}) is previously obtained by homomorphic encryption of a second aggregation (M2) of a plurality of second encodings of a plurality of data items of a second group in a second memory (DM2). Including, the first and second aggregations are pre-obtained by forming respective at least two-dimensional (2D) matrices from the first encoding and the second encoding; The method, wherein the processing includes matrix multiplication of the at least two matrices by a processor capable of single instruction multiple data SIMD processing.

10. A program which, when executed by one or more computing devices, causes said one or more computing devices to perform the method of any one of claims 1 to 9.

11. 1. A system for facilitating data similarity queries across a network of data memories (DMS), the system comprising: receiving a first ciphertext ({M1}) for a first aggregation (M1) of a plurality of first encodings of a first group of a plurality of data items from a first memory (DM1) in the network; receiving a second ciphertext ({M2}) for a second aggregation (M2) of the second encodings of the second group of data items from a second memory (DM2) in the network; Processing the first and second ciphertexts into a third ciphertext ({M1*M2}) that can be decrypted to a plaintext output (M1*M2). wherein the plaintext output indicates or is convertible to a similarity measure between data items from the two groups; the first and second aggregations are pre-obtained by forming respective at least two-dimensional (2D) matrices from the first encoding and the second encoding; The processing includes matrix multiplication of the at least two matrices by a processor capable of single instruction multiple data SIMD processing.

12. 1. A system for facilitating data similarity queries across a network of data memories (DMS), comprising: i) forming a first aggregation (M1) from a plurality of first encodings of a first group of a plurality of data items in a first memory (DM1); and ii) forming a second aggregation (M2) from a plurality of second encodings of a second group of a plurality of data items in a second memory (DM2); The two aggregates are homomorphically encrypted into two ciphertexts ({M1}) and ({M2}). It is configured as follows: the first and second aggregations are obtained by forming respective at least two-dimensional (2D) matrices from the first encoding and the second encoding; The two ciphertexts are processed into a third ciphertext ({M1*M2}), the processing including a matrix multiplication of the at least two matrices by a processor capable of single instruction multiple data SIMD processing.

13. 1. A system for facilitating data similarity queries across a network of data memories (DMS), the system comprising: configured to decrypt a third ciphertext ({M1*M2}), previously obtained by processing the first ({M1}) and second ({M2}) ciphertexts, into a plaintext output (M1*M2) that can be converted into a similarity measure of data items across groups; said first ciphertext ({M1}) is obtained in advance by homomorphic encryption of a first aggregation (M1) of a plurality of first encodings of a plurality of data items of a first group in a first memory (DM1), and said second ciphertext ({M2}) is obtained in advance by homomorphic encryption of a second aggregation (M2) of a plurality of second encodings of a plurality of data items of a second group in a second memory (DM2); the first and second aggregations are pre-obtained by forming respective at least two-dimensional (2D) matrices from the first encoding and the second encoding, and the processing includes matrix multiplication of the at least two matrices by a processor capable of single instruction multiple data SIMD processing.

Images