Verifiable secure dataset operations using private federated keys

Abstract

To perform a federated operation, a module executing in a trusted execution environment (TEE) receives a first data set including individually identifiable information (PII) data and non-PII data from a first party (1P) data source. The module is used for preprocessing the PII data to generate first formatted PII data, and the first formatted PII data conforms to a predefined format; matching, in the TEE, the first formatted PII data with second formatted PII data included in a second data set; performing a joint operation between the first data set and the second data set based on the match to generate a joint data set; and providing the federated data set to a data service operating independently of the IP data source.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of the filing date of U.S. Provisional Patent Application No. 63 / 391,794, entitled “VERIFIABLE SECURE DATASET JOININGWITH PRIVATE JOIN KEYS,” filed on July 24, 2022. The entire contents of the provisional application are hereby expressly incorporated herein by reference. Technical Field

[0003] The present disclosure relates to secure computing environments, and more particularly, to techniques implemented in a cloud or another suitable environment for improving data security and computing efficiency when performing operations such as joining data sets from multiple parties. Background Art

[0004] The background description provided herein is for the purpose of generally presenting the background of the present disclosure. The work of the presently named inventors (to the extent that it is described in this background section) and aspects of this description that may not be identified as prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art to the present disclosure.

[0005] Today, some services or applications may attempt to join data sets from different independent parties. Data sets often include data that one party does not wish and / or is not allowed to share with another party, which for simplicity may be referred to as "restricted data." An example of such restricted data is personally identifiable information (PII). This data may not be easily removed before performing a join operation, as it may operate as a join key, i.e., data that logically links records in separate data sets.

[0006] For example, a data service DS1 may store free device identifiers IDs at different times. d1 、ID d2 、…ID N The second data service DS2 may maintain readings of temperature sensors from a set S1 of identified devices, and the second data service DS2 may maintain readings of pressure sensors from a set S2 that at least partially overlaps with set S1. It may be desirable to join the temperature and pressure readings of the intersection of sets S1 and S2 without revealing the identity of the device corresponding to a particular sensor reading.

[0007] It is desirable to provide a computing environment in which federated operations on data sets from multiple sources can be performed securely and efficiently. Summary of the invention

[0008] The techniques of the present disclosure support federated operations on data sets that eliminate the need for a source of first-party data (1PD) to reveal sensitive data such as PII to another party, without requiring the 1PD source (or "customer data source") to perform computationally expensive hashing and / or encryption locally, or to hand over the data to another party for these operations.

[0009] Using the techniques of the present disclosure, the system can ensure that it is sufficient for a customer data source to connect only to a secure connector operating in a trusted execution environment (TEE) to provide data, and that the secure connector does not provide access to customer data to any other party. These techniques also allow the customer data source to not share credentials with modules other than the secure connector.

[0010] The secure connector can receive 1PD and encrypt at least part of the 1PD, such as PII fields. The encrypted data then flows securely through an extract-transform-load (ETL) pipeline to the PII matching module, which is also implemented in the TEE. Only certified security code can access the cryptographic keys required to decrypt the encrypted fields, and no party can extract sensitive information from the encrypted PII or modify the functionality of the secure connector or PII matching module. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram of an example computing environment in which at least some techniques of the present disclosure may be implemented;

[0012] Figure 2A It shows that Figure 1 A block diagram of an example computing architecture including a security control plane and a data plane utilized in A's computing environment;

[0013] Figure 2B is shown similar to Figure 2A A block diagram of another example of a computing architecture of , except that here the environment includes additional infrastructure for managing cryptographic keys and privacy budgets;

[0014] Figure 3A is a block diagram of an example pipeline for performing a safe join operation on a 1PD with another data set, which example pipeline can be found at Figure 1 A's computing environment or another suitable environment;

[0015] Figure 3B is roughly similar to Figure 3A A block diagram of a pipeline of the pipeline of FIG. 1 , but wherein the security connector and the PII matching module are combined into a single entity;

[0016] Figure 4A is available in Figure 3A or Figure 3BA flowchart of an example method for ingesting plaintext 1PD, preprocessing the 1PD, and re-encrypting the 1PD in a secure connector implemented in an environment of FIG.

[0017] Figure 4B is roughly similar to Figure 4A A flowchart of an example method of an example method of , but wherein at least a portion of the 1PD arrives at the secure connector in an encrypted format;

[0018] Figure 4C is roughly similar to Figure 4A A flowchart of an example method of an example method of, but wherein the secure connector hashes the PII in the 1PD before sending the 1PD to the PII matching module;

[0019] Figure 5A is available in Figure 3A or Figure 3B A flowchart of an example method in a PII matching module implemented in the depicted environment for receiving an IPDS with pre-processed and encrypted PII from a secure connector, decrypting the PII, and matching the IPDS with another data set using PII fields;

[0020] Figure 5B is available in Figure 3B The environment depicted in the implementation is roughly similar to Figure 5A A flowchart of an example method of an example method of, but wherein the PII matching module performs the matching using hashed PII fields or values;

[0021] Figure 5C is available in Figure 3A or Figure 3B A flowchart of an example method for generating a federated data set for a data service in a PII matching module implemented in an environment of;

[0022] Fig. 6A is a flow chart of an example method in a customer data source for providing 1PD in plain text to a secure connector;

[0023] Figure 6B is a flow chart of an example method in a customer data source for encrypting PII fields in an 1PD and providing the 1PD to a secure connector;

[0024] Fig. 7A It shows that according to Figure 4A , Figure 5A and Fig. 6A When 1PD travels through Figure 3A or Figure 3B Block diagram of the conversion of PII and non-PII data in 1PD in the context of

[0025] Figure 7BIt shows that according to Figure 4A , Figure 5A and Figure 6B When 1PD travels through Figure 3A or Figure 3B A block diagram of the conversion of PII and non-PII data in 1PD in the context of

[0026] Figure 7C It shows that according to Figure 4B , Figure 5B and Fig. 6A When 1PD travels through Figure 3A or Figure 3B Block diagram of the conversion of PII and non-PII data in 1PD when the environment is in place. DETAILED DESCRIPTION

[0027] As discussed in more detail below, the secure connector and PII matching module can be executed in the TEE to securely and efficiently perform federation operations on data sets from different parties. In some implementations, the secure connector also performs pre-processing of PII so that PII from different data sets have the same format to allow efficient matching operations. The secure connector, PII matching module, and components of the ETL pipeline can be implemented in a cloud computing environment, or simply "the cloud."

[0028] These components allow the burden of data obfuscation (including hashing and / or encryption) to be shifted from the customer data source to the cloud, while protecting PII from inspection by other parties. Implementing these modules in the TEE allows the PII matching module to perform matching and / or joining in plaintext, but guaranteeing end-to-end privacy of PII and integrity of data processing.

[0029] These techniques address technical issues associated with existing approaches, such as the inability of 1P data owners to maintain control of their datasets and prevent other parties from accessing individual-level PII, or the need for 1P data owners to share PII with various intermediaries (e.g., services that apply analytics to 1PD). Even when data owners hash PII fields to obfuscate certain information, and then a platform uses the hashed fields as a federation key to associate or federate datasets, these approaches are computationally heavy because data often must conform to a specific format to be properly ingested. Since there are often many 1PD sources, hashing and comparing multiple different formats leads to inefficiencies and even errors.

[0030] As a more specific example, a telephone number may be in a format such as '555.555.5555', '555-555-5555', or '(555)555-5555', each of which corresponds to a different hash. Mailing addresses result in an even greater variety of formats. Furthermore, while hashing provides obfuscation, hashed data has security vulnerabilities, such as, for example, exposure to dictionary attacks.

[0031] These techniques are applicable to a wide variety of applications, including, for example, the ad tech industry, where advertisers measure the effectiveness of ad campaigns by determining which consumer groups or audiences buy certain types of products or determining which ads result in the highest sales. To do this, the system may combine IP D (e.g., sales data in a customer relationship management (CRM) system) and ad campaign data (e.g., information about people who interact with an ad). Because both IP D and campaign data contain PII, such as phone numbers, IP addresses, email addresses, physical addresses, etc., the PII may be used as a joint key.

[0032] First reference Figure 1 , Figure 2A and Figure 2B Discusses an example environment suitable for implementing such techniques. Then, reference Figure 3A and Figure 3B Consider an example pipeline for matching / joining operations, followed by a discussion of example approaches in secure connectors, PII matching modules, and IP data sources.

[0033] Computational environment for secure multi-party computation and federated operations

[0034] The secure control plane described herein (sometimes referred to herein as "SCP") provides an unobservable secure execution environment in which services can be deployed. In particular, any business logic (e.g., application code) that provides a service can be executed within the secure execution environment to provide the security and privacy guarantees required for the workflow, while no party can observe the runtime computations. The state of the environment is opaque even to the administrator of the service, and the service can be deployed on any supported cloud.

[0035] As an example, two clients generating data, Client 1 and Client 2, may wish to combine the data streams they receive from their respective clients so that the clients can generate quantitative metrics related to those clients that could not be derived from their respective data sets. As a more specific example, for example, Client 1 may be a retailer having data indicative of customer transactions, and Client 2 may be an analytics engine capable of measuring the effectiveness of an advertising campaign for products offered by the retailer.

[0036] Client 2 may offer a service with an algorithm that Client 2 claims will securely perform data analysis. However, Client 1 may not wish to expose its customer data to Client 2 in a manner that would potentially allow the data to be leaked or used in a manner that does not comply with Client 1's privacy and security guarantees. Therefore, Client 1 wants to ensure that (1) its customer data cannot be leaked by Client 2 or any other party, and (2) the logic used to analyze the customer data complies with Client 2's security requirements. The technology disclosed herein provides a secure execution environment in which business logic executes, such that sensitive data analyzed by the business logic remains encrypted everywhere except within the secure execution environment, and provides proof so that any party can ensure that logic running within the secure execution environment executed as guaranteed.

[0037] In general, services that perform calculations (i.e., use business logic to process events or requests) are split between a data plane (DP) and a security control plane (SCP). The business logic specific to the calculation is hosted within the DP, where the DP is within the TEE, also referred to herein as an enclave. The business logic can be provided to the DP as a container, where the container is a software package containing all the necessary elements for running the business logic in any environment. For example, the container can be provided to the SCP by the business logic owner. Functionally, the SCP provides a secure execution environment and facilities to deploy and operate the DP on a large scale, including managing cryptographic keys, buffering requests, tracking privacy budgets, accessing storage, and orchestrating policy-based horizontal automatic expansion. The SCP execution environment isolates the DP from the details of the cloud environment, thereby allowing the service to be deployed on any supported cloud provider without changing the DP. Both the DP and the SCP work together by communicating via an input / output (I / O) application programming interface (API) (also referred to herein as a control plane I / O API or CPIO API).

[0038] In one example implementation, all data traversing the SCP is always encrypted, and only the DP has access to the decryption key. For example, for a particular service, the business logic may include performing event aggregation and outputting an aggregate summary report. In such an instance, the SCP delivers encrypted requests from one or more event sources to the DP, which promptly decrypts the request, processes the request, checks the privacy budget, and generates and sends an encrypted report. In addition, when outside the DP, the decryption key can be bit-split so that only the DP can assemble the decryption key within the TEE. Depending on the desired application, the output from the DP can be edited or aggregated in a way that the output can be shared and no individual user's data can be identified or leaked.

[0039] SCP provides several privacy, trust, and security guarantees. Regarding privacy, services using SCP can provide assurances that no stakeholder (e.g., devices operated by the client, cloud platform, third party) can act alone to access or leak plaintext (i.e., non-encrypted) sensitive information, including administrators deployed by SCP. In addition, regarding trust, DP runs in a secure execution environment in a trusted state when the enclave is booted. For example, SCP can be implemented on a trusted platform module (TPM) or a virtual trusted platform module (vTPM) according to secure boot standards and / or using a trusted and / or certified operating system (OS). Starting from an audited code base and reproducible builds, cryptographic proofs are used to prove DP binary identity and provenance at runtime (as will be discussed in more detail below). In addition, the key management service (KMS) only releases cryptographic keys to verified enclaves. Therefore, any tampering with the DP image will result in the system being unable to decrypt any data. Cloud providers are absolutely trustworthy, given the strong incentives they have to guarantee their terms of service (ToS) guarantees. Regarding security, the secure execution environment is unobservable. The secure execution environment's memory is encrypted or otherwise hardware protected to prevent access from other processes. In an example implementation, core dumps are not possible. All data is encrypted in transit and at rest, and all I / O to / from the DP is encrypted. No one has access to the private key in plain text (e.g., the KMS is locked, the key is split, and the key is only available within the DP, which is within the secure execution environment.

[0040] SCP distributes trust in a way that three stakeholders need to cooperate in order to leak plaintext user event data. SCP also uses a distributed trust model to ensure that two stakeholders need to cooperate to tamper with the privacy budget service. Distributed trust works using both event decryption and the privacy budget service. Regarding event decryption, the private key required to decrypt the events received at the SCP is generated in a secure environment and bit-split between at least two KMSs, each of which is under the control of an independent trusted party. The KMS is configured to release key material only to DPs that match a specific hash. If the DP is tampered with, their keys will not be released. In such a scenario, the service can be started, but no events will be decrypted. Similarly, the privacy budget service can be distributed between two independent trusted parties, and transaction semantics can be used to ensure that the budgets of the two trusted parties match, which allows detection of budget tampering.

[0041] If you will refer to Figure 2BAs discussed, SCP also provides a mechanism for proving that any business logic running on DP corresponds to publicly released code, thereby allowing other parties to verify the business logic used to analyze sensitive data. The complete code base of the business logic (except for the scenario involving proprietary business logic described with reference to Figure 5) is available for inspection and audit by all stakeholders. The build is reproducible, and any stakeholder can build a DP container. Building a deployable image generates a set of encrypted hashes (e.g., platform configuration registers (PCRs)). Therefore, all parties can verify whether the deployed product matches the published code base by comparing PCRs. After building the logic, DP provides PCRs (e.g., via CPIO API) to parties requesting verification of the built logic. For example, KMS is configured to release key material only to images that match PCRs generated from building the published logic. This ensures that the private key used to decrypt sensitive information is only available for images corresponding to specific submissions to specific repositories.

[0042] Turning to an example computing system in which the SCP of the present disclosure may be implemented, Figure 1 An example computing system 100 is shown. Computing system 100 includes a client computing device 102 (also referred to herein as client device 102) coupled to a cloud platform 122 (also referred to herein as cloud 122) via a network 120. Network 120 may typically include one or more wired and / or wireless communication links, and may include, for example, a wide area network (WAN) (such as the Internet), a local area network (LAN), a cellular telephone network, or another suitable type of network or combination of networks. Although examples of the present disclosure are primarily directed to cloud-implemented architectures, it should be understood that the techniques disclosed herein, including techniques for providing a secure execution environment in which sensitive data is processed, techniques for generating, partitioning, and distributing keys, and techniques for providing mechanisms for authenticating proprietary business logic, may also be applied in non-cloud systems.

[0043] For example, client device 102 may be a portable device such as a smart phone or a tablet computer. Client device 102 may also be a laptop computer, a desktop computer, a personal digital assistant (PDA), a wearable device such as smart glasses, or other suitable computing devices. Client device 102 may include memory 106, one or more processors (CPU) 104, a network interface 114, a user interface 116, and an input / output (I / O) interface 118. Client device 102 may also include Figure 1Components not shown in the diagram, such as a graphics processing unit (GPU). The client device 102 may be associated with a service user, which is an end user of the services provided by the SCP, as described below. The end user operates the client device 102 (or more specifically, a browser or application on the client device 102) that sends a request / event to the service. In order to send a request or event to the service, the client device 102 encrypts the request / event using a public key, which the client device 102 may retrieve from a public key repository (e.g., a public key repository server 178). The client device 102 is exemplary only. As discussed below, the cloud platform 122 may receive incoming events and / or requests from the client device 102, from a browser / application / client process executing on the client device 102, or from another computing device that issues a request on behalf of the client device 102 or forwards a request from the client device 102. In addition, although in Figure 1 Only one client device is shown in FIG. 1 , but computing system 100 may include multiple client devices capable of communicating with cloud platform 122 .

[0044] The network interface 114 may include one or more communication interfaces, such as hardware, software, and / or firmware, for enabling communication via a cellular network, a WiFi network, or any other suitable network, such as the network 120. The user interface 116 may be configured to provide information to a user, such as a response to a request / event received from the cloud platform 122. The I / O interface 118 may include various I / O components (e.g., ports, capacitive or resistive touch-sensitive input panels, keys, buttons, lights, LEDs). For example, the I / O interface 118 may be a touch screen.

[0045] The memory 106 may be a non-transitory memory and may include one or more suitable memory modules, such as a random access memory (RAM), a read-only memory (ROM), a flash memory, other types of permanent memory, etc. The memory 106 may store machine-readable instructions that can be executed on one or more processors 104 and / or special processing units of the client device 102. The memory 106 also stores an operating system (OS) 110, which may be any suitable mobile or general-purpose OS. In addition, the memory 106 may store one or more applications that transmit data with the cloud platform 122 via the network 120. Transmitting data may include sending data, receiving data, or both. For example, the memory 106 may store instructions for implementing a browser, an online service, or an application that requests data from / sends data to an application (i.e., business logic) implemented on a DP of a secure execution environment on the cloud platform 122, as described below.

[0046] The cloud platform 122 may include multiple servers associated with a cloud provider to provide cloud services via the network 120. The cloud provider is the owner of the cloud platform 122 on which the SCP 126 is deployed. Figure 1 Only one cloud platform is shown in FIG. 1 , but SCP 126 can be deployed on multiple cloud platforms, even if the cloud platforms are operated by different cloud providers. The servers that provide cloud platform 122 can be distributed across multiple sites to improve reliability and reduce latency. Individual servers or groups of servers within cloud platform 122 can communicate with client devices 102 and with each other via network 120. Example servers that may be included in cloud platform 122 are discussed in further detail below. Although not specifically described herein, Figure 1 106. Each server in the cloud platform 122 is shown, but each server included in the cloud platform 122 may include one or more processors similar to processor 104, which are suitable and configured to execute various software stored in one or more memories similar to memory 106. The server may also include a database, which may be a local database stored in the memory of a particular server or a network database stored in a network-connected memory (for example, in a storage area network). The server may also include a network interface and an I / O interface similar to interfaces 114 and 118, respectively. In addition, it should be understood that although certain components are described as separate servers, in general, the term "server" may refer to one or more servers. In addition, although functions are generally described as being performed by separate servers, some functions described herein may be performed by the same server.

[0047] The cloud platform 122 includes an SCP 126, which includes a TEE 124. TEE 124 is a secure execution environment in which DP 128 is isolated. A TEE, such as TEE 124, is an environment that provides execution isolation and provides a higher level of security than a conventional system. TEE 124 can use hardware to implement isolation (known as confidential computing). The cloud provider is considered to be the root of trust for SCP 126 and complies with the terms of service (ToS) agreement of the cloud platform 122. The hardware manufacturer of the server that provides TEE 124 also has a ToS guarantee and therefore also provides an additional layer of trust. SCP 126 also uses technology to ensure that the state at startup is secure, including using a minimized OS image recommended by the cloud provider, and using a secure boot sequence based on TPM / vTPM to enter the OS image.

[0048] One or more servers of the cloud platform 122 perform control plane (CP) functions (i.e., support SCP 126), and one or more servers perform data plane (DP) functions. All functions of DP 128 are performed by servers within TEE 124. TEE 124 can be deployed and operated by an administrator. The administrator can audit the logic to be implemented on DP 128 and verify against the hash of the binary image to deploy logic 142. On CP, there may be a front-end server 134, which receives external request / event indications (e.g., from client device 102), buffers requests / events until they can be processed by DP 128, and forwards received requests to DP 128. Generally speaking, as used herein, unless otherwise specified, a request may also refer to an event, or may include one or more events. In some implementations, there is a third-party server 136 between the client device 102 and SCP 126. The third-party server 136 (which may include one or more servers and may or may not be hosted on the cloud platform 122) may be responsible for receiving requests from the client device 102 (which are encrypted by the client device 102) and later dispatching the encrypted requests to the SCP 126. In some cases, the third party is the administrator of the service. The third-party server 136 does not have the key to decrypt the requests. The third-party server 136 may, for example, aggregate the requests into batches and store these batches (e.g., on the cloud storage 160). The third-party server 136 or the cloud storage server 160 may notify the front-end server 134 that the request is ready to be processed, and / or the front-end server 134 may subscribe to notifications that are pushed to the front-end server 134 when a batch is added to the cloud storage 160.

[0049] DP 128 includes a server (which may include one or more servers) including one or more processors 138 (similar to processor 104) and one or more memories 140 (similar to memory 106). Memory 140 includes business logic 142 (also referred to as logic 142), which can be executed by processor 138. Business logic 142 is used to implement any application or service deployed on TEE 124. Memory 140 can also store key cache 146, which stores cryptographic keys for encrypting and decrypting communications. In addition, memory 140 includes CPIO API 144, which includes a function library for communicating with other elements of cloud platform 122 (including components on CP of SCP 126). CPIO API 144 can be configured to interface with any cloud platform provided by a cloud provider. For example, in a first deployment, SCP 126 can be deployed to a first cloud platform provided by a first cloud provider. DP 128 hosts specific business logic 142, and CPIO API 144 facilitates communication between logic 142 and the first cloud platform. In a second deployment, SCP 126 may be deployed to a second cloud platform provided by a second cloud provider. DP 128 may host the same business logic 142 as the first deployment, and CPIO API 144 is configured to facilitate communication between logic 142 and the second cloud platform. Thus, SCP 126 may be deployed to different cloud platforms without editing the underlying business logic 142, and only configuring CPIO API 144 to interface with a specific cloud platform.

[0050] There may be additional CP-level services provided by servers of cloud platform 122 that support SCP 126. For example, verifier server 148 may implement a verifier module capable of verifying whether business logic 142 complies with security policies, as discussed below with reference to FIG. 5. As another example, privacy budget service server 152 may implement a privacy budget service that verifies whether a user or device's privacy budget has been exhausted. As discussed below with reference to FIG. Figure 2B As discussed, one or more privacy budget services may additionally or alternatively be implemented by a trusted party.

[0051] In addition, the cloud platform 122 may include other servers and databases that communicate with the SCP 126, as described in the following paragraphs. These servers may facilitate the CP functions of the SCP 126. In particular, the CP functions may be distributed across several servers, as will be discussed below. However, the DP 128 remains within the TEE 124 and is not distributed outside the TEE 124.

[0052] As described above, cloud storage 160 may store encrypted batches of requests before the encrypted batches are received by front-end server 134. Cloud storage 160 may also be used to store responses after DP 128 has processed received requests, or to perform storage functions for other components of cloud platform 122. Queue 162 may be used by front-end server 134 to store pending requests before they can be analyzed by DP 128. For example, after receiving a request from client device 102, front-end server 134 may receive the request and temporarily store the pending request in queue 162 until DP 128 is ready to process the request. As another example, after receiving notification from third-party server 136 that a batch of requests is stored in cloud storage 160, front-end 134 may retrieve the batch of requests and place the batch of requests in queue 162, where the batch of requests awaits analysis by DP 128.

[0053] Key Management Server (KMS) 164 provides a KMS that generates, deletes, distributes, replaces, rotates, and otherwise manages cryptographic keys. Trusted Party 1 Server 166 and Trusted Party 2 Server 172 are servers associated with Trusted Party 1 and Trusted Party 2, respectively, that provide the functionality of each trusted party. Figure 1 Only two trusted parties are shown, but the cloud platform 122 may include multiple trusted parties. Each trusted party can manage a privacy budget and can also audit the logic 142 implemented on the DP 128 to verify the built product against the hash of the published logic. The trusted party has the creation and management of asymmetric keys for encrypting and decrypting user data. The trusted party can securely generate keys and publish public keys to the world. As will be discussed in detail with reference to FIG. 4, the private key can be bit-split into two parts (one split under the control of each trusted party, but any number of N splits can also be supported, such as when there are N trusted parties). Envelope encryption technology can be used, in which each trusted party encrypts the split of each key with the symmetric key of the KMS and stores the encrypted split in its repository. Envelope encryption allows the rotation of the envelope without having to rotate the keys within the envelope. The public key can be stored and managed by the public key repository server 178. Additionally or alternatively, the KMS server 164 can manage the public key.

[0054] The computing system 100 may also include a public security policy storage 180, which may be located on or off the cloud platform 122. The public security policy storage 180 stores security policies so that the security policies are accessible to the public (e.g., by the client device 102, by components of the cloud platform 122). A security policy (also referred to herein as a policy) describes what actions or fields are allowed for the output of a composition service. A policy may also be described as a machine-readable and machine-implementable privacy design document (PDD). The policy will be further described with reference to FIG5.

[0055] Next reference Figure 2A , example architecture 200A illustrates connections between components and software elements of computing system 100. Client device 102 may retrieve a public key (e.g., from public key repository server 178) in order to address a request to a service being implemented on DP 128 (i.e., by business logic 142). For example, client device 102 may initiate a request to access content provided by a service, or may publish an event including user behavior data.

[0056] The encrypted request from the client device 102 is first received by the front-end module 234 of the SCP 126 (i.e., a module implemented by the front-end server 134). In some implementations, the request is first received by a third party that batches the request before notifying the front-end 234 (or causing the front-end 234 to be notified). In such cases, the front-end 234 may retrieve the encrypted request from the cloud storage 160. In any case, the front-end 234 passes the encrypted request to the DP 128 using the functions defined by the CPIO API 144. The front-end 234 may store the encrypted request in the queue 162 until the DP 128 is ready to process the request and retrieve the request from the queue 162. The DP 128 decrypts the request and processes the request according to the business logic 142. Decrypting the request may include communicating with the KMS 264 (i.e., a cloud KMS implemented by the KMS server 164) to retrieve and assemble a private key for decrypting the request, and / or communicating with a trusted party, such as Figure 2B shown.

[0057] Processing the request may include communicating with a privacy budget service 252 (e.g., implemented by a privacy budget service server 152) using CPIO API 144 functionality to check the privacy budget and ensure compliance with the privacy budget. The privacy budget tracks processed requests and events. For example, there may be a maximum number of requests originating from a particular user that may be processed during a particular computation or time period. Ensuring compliance with the privacy budget prevents parties analyzing the output from DP 128 from extracting information about a particular user. By checking whether the privacy budget is being adhered to, DP 128 provides a differentially private output.

[0058] The results from processing these requests may be encrypted by DP 128 and may be edited and / or aggregated so that the output does not reveal information about a particular user. DP 128 may store the results, for example, in cloud storage 160, where the results may be retrieved by parties having a decryption key for the results. As an example, if the results are processed for a third-party server 136, DP 128 may encrypt the results using a key that the third-party server 136 may decrypt.

[0059] Go to Figure 2B, architecture 200B is similar to architecture 200A, except that additional details regarding key management and privacy budget are shown. Figure 2A compared to, Figure 2B Also shown are trusted party 1 server 166 (referred to herein as trusted party 1 166 for simplicity), trusted party 2 server 172 (referred to herein as trusted party 2 172 for simplicity), and public key distribution service 278. Public key distribution service 278 provides public keys to client devices 102, which can use these public keys to address requests to DP 128, front end 234, or third-party servers 136 (which aggregate these requests). Figure 2B 172). The public key distribution service 278 may be operated by the public key repository server 178 or by the KMS server 164. Trusted party 1 166 includes a key cache 268 containing an encrypted split-1 key (i.e., the encrypted first portion of the private key), and trusted party 2 172 includes a key cache 274 containing an encrypted split-2 key (i.e., the encrypted second portion of the private key). Each of the trusted parties 166, 172 may also provide a privacy budget service 270, 276, and may each manage an instance of a privacy budget. Distributing the management of the privacy budget to the two trusted parties helps ensure that no one trusted party can tamper with the privacy budget. The privacy budget services 270, 276 should both implement the same privacy budget; therefore, if the two services return different outputs, the SCP 126 may recognize that one of the trusted parties 166, 172 has tampered with the privacy budget. Figure 2B The architecture shown in prevents any one trusted party from having full control over private decryption keys or privacy budgets. A single trusted party cannot act alone to provide an unlimited budget to any user, and therefore a single trusted party cannot repeatedly aggregate the same batch of data.

[0060] Example pipeline for performing safe match / union operations

[0061] Next, Figure 3A A pipeline 300A is shown that can be implemented at least in part in the environment discussed above. Pipeline 300A receives a data set from an IP data source 302 and provides the results of the match / union to a data service 304. The parties controlling the control systems 302 and 304 are separate and independent, and it is expected that the data service 304 performs operations (e.g., analysis) using the data set from the IP data source 302 without relying on or being able to access the data included in the data set, particularly PII. The IP data source 302 can be any suitable external source of IP data keyed by plain text PII or any other suitable data. The IP data source 302 can be, for example, a CRM, a proprietary system, a file available on the Internet, etc.

[0062] The IP data source 302 provides a data set to a secure connector 320 implemented in the cloud 310 via an encrypted link 303. Link 303 can be, for example, an SSL / TLS connection established over the Internet. The secure connector 320 can operate in an audited and certified TEE. As discussed in more detail below, the secure connector 320 can hash and / or encrypt some or all of the received data sets in operation. The secure connector 320 provides the hashed / encrypted data set to the ETL pipeline 324 via an encrypted link 322. The ETL pipeline 324 can move the data set to a data repository 330 or a secure federation module 328 via an encrypted link 326. The ETL pipeline 324 can typically perform data conversion and field mapping to conform to a certain pattern and format non-encrypted fields. The repository 330 can be a data storage service that allows time-delayed consumption of data ingested from the IP data source 302.

[0063] Similar to the secure connector 320, the PII matching module 328 may operate in an audited and certified TEE. The PII matching module 328 may match and join an IP data set with another data set, which may, for example, come from another IP data source or may be inside the data service 304. The PII matching module 328 then provides a privacy-secure output to the data service 304, which may operate on a cloud platform 312 or any other suitable platform.

[0064] like Figure 3A As shown, ETL pipeline 324 can deliver data to secure federation module 328 for immediate consumption by data service 304, or to repository 330. Repository 330 supports a "once ingested, multiple uses" workflow. Repository 330 always stores sensitive PII in a hashed or encrypted form. In some implementations, additional layers such as static encryption further ensure the security of data stored in repository 330.

[0065] refer to Figure 3B , pipeline 300B is similar to pipeline 300A, but here a single component 325 operating in the TEE implements the functionality of both the secure connector 320 and the PII matching module 328. However, this simplified architecture does not support the storage of encrypted PII in the repository 330 for later consumption.

[0066] Overall reference Figure 3A and Figure 3B, one or more TEEs supporting the secure connector and PII matching modules are services with provable security and privacy features. More specifically, these services ensure that parties can verify that they are connected to the correct server; that parties can verify what the TEE box does by inspecting the code repository; and that parties can verify that the repository code accurately corresponds to the image running in the server. In addition, the attestation infrastructure in the cloud provider 310 guarantees that the required decryption keys can only be used within the TEE with a specific signature.

[0067] Example workflow for performing a secure match / union operation

[0068] Next, refer to 4A to 7C discuss Figure 3A and Figure 3B Several example workflows that the pipeline can support. 4A to 6B The methods of may be implemented using suitable processing hardware, for example, as a set of software instructions stored on a non-transitory computer-readable medium and executable by one or more processors.

[0069] First reference Figure 4A , method 400A may be implemented in secure connector 320 or 325. Method 400A includes plaintext PII matching, server-side encryption, and use of customer-generated keys. Method 400A begins at block 403, where the secure connector performs authentication with the IP data source. More specifically, the secure connector may initiate a connection between an IP data source (e.g., IP data source 302) and the secure connector. The customer may first provide encrypted credentials to ensure that the secure connector is the only entity that can connect to the IP data source. KMS 164 (see Figure 1 , Figure 2A and Figure 2B ) can use decryption keys for credentials under a customer-owned account, and KMS 164 ensures that only the secure connector 320 can perform decryption operations using those credentials.

[0070] The secure connector can decrypt the credentials and use the decrypted credentials to authenticate with the IP data source. Data transmission occurs over SSL / TLS or a similar protocol that allows authentication of endpoints. In some cases, the secure connector and the IP data source can use mutual authentication (mTLS) to ensure both ends of the connection that data flows from and to the intended endpoint. Some IP data sources require reuse of credentials, while other IP data sources rely on tokens, certificates, or another technology to obtain data over a secure connection. According to another implementation, the secure connector and the IP data source use certificates and encryption schemes instead of credentials to provide access to data. The certificates required for the connection are encrypted, and are used in such a way that only the secure connector has the certificates that can be used to establish a successful connection.

[0071] In any case, the customer associated with the IP data source generates a data encryption key DEK and a key encryption key (KEK) locally using the cloud KMS discussed above. The customer's computing system can encrypt the DEK using the KEK using the API of the cloud KMS. The customer also configures the KMS to allow the security connector and the PII matching module to decrypt the KEK. At box 404, the security connector receives the encrypted DEK associated with the IP data source. At box 405, the security connector provides the encrypted DEK to the PII matching module.

[0072] At block 410, the secure connector ingests a data set in plain text from an IP data source. Fig. 7A As shown for further clarity in FIG. 4 , according to the workflow, the data set at stage 702A includes both non-PII and PII fields in plain text. At box 420, the secure connector pre-processes the PII to match a specific standard format. This conversion may increase the match rate at the PII matching module, and thus reduce the error rate and improve efficiency.

[0073] At block 422, the secure connector decrypts the DEK using the KMS and encrypts at least the PII field of the ingested data set with the DEK (see Fig. 7A , stage 704A). At block 430, the secure connector sends the data to the PII matching module via the pipeline for matching with another data set based on the PII. Figure 5A As discussed, the PII matching module may perform matching and joining in plain text. Fig. 7A Such a plaintext comparison is shown at stage 706A.

[0074] Next, Figure 4B Method 400B is shown. Similar blocks are labeled with similar reference numerals, and only the differences between method 400A and method 400B are considered below. Method 400B includes plaintext PII matching and client-side encryption. At block 411, the secure connector ingests (e.g., obtains) a data set from an IP data source, and in this case, the data set includes encrypted PII, also as shown. Figure 7B As shown in stage 702A.

[0075] Figure 4C Method 400C is shown, which includes PII matching using hashing. Similar boxes are labeled with similar reference numerals, and only the differences between method 400A and method 400B are considered next. Method 400B includes plaintext PII matching and client-side encryption. At box 421, the secure connector hashes the PII field, and at box 432, the secure connector provides a data set with the hashed PII field to the PII matching module for hash-based comparison. Figure 7CIt is shown that at stage 702C, the data set includes plaintext PII and non-PII data; at stage 704C, the PII is pre-processed and hashed; and at stage 70BC, the comparison is based on the formatted / processed and hashed PII.

[0076] Figure 5A is a flow chart of an example method 500A in a PII matching module, such as PII matching module 325. Method 500A may correspond to method 400A or 400B in a secure connector.

[0077] At block 501, the PII matching module receives a data set with pre-processed and encrypted PII from a secure connector via an encrypted link (see Fig. 7A , stage 704A). At block 510, the PII matching module uses the KMS to decrypt the data using the DEK, and then at block 520, decrypts the encrypted PII field using the DEK.

[0078] At block 530, the PII matching module matches the 1PD dataset with another dataset (such as an internal dataset) based on the PII fields (see Fig. 7A , stage 706A). The PII matching module may also discard all non-matching rows. At block 540, the PII matching module may provide the matched data set to a data service, such as data service 304.

[0079] Figure 5B 5 is a flow chart of another example method 500B in a PII matching module such as PII matching module 328 or 325. Method 500B may correspond to method 400C in a secure connector. At block 502, the PII matching module receives a data set with pre-processed and hashed PII from a secure connector. At block 531, the PII matching module may match the data set with another data set based on the hashed PII fields and discard non-matching rows. At block 540, the PII matching module may provide the matched data set to a data service such as data service 304.

[0080] Figure 5C is a flow chart of an example method 500C for generating a federated data set for a data service in a PII matching module. At block 550, the PII matching module may use PII in non-hashed or hashed format to determine a match between data sets in accordance with methods 500A and 500B, respectively.

[0081] At block 560, the PII matching module may map external identifiers of matching rows of the data set to internal identifiers. At block 570, the PII matching module may also augment each row of the output data set with metadata indicating the type of match that occurred (e.g., based on email, phone, address) for post-processing (e.g., conflict and duplicate resolution). At block 572, the PII matching module may remove all PII from the output data set.

[0082] In addition to or in lieu of block 560, the PII matching module at block 562 may generate a list of internal identifiers that match between the data sets. Flow may also proceed to block 570, where the PII matching module augments each row with metadata, as discussed above. Further, in addition to or in lieu of blocks 560 and 562, the PII matching module at block 564 may include any combination of fields from both the data sets and / or metadata, but without any PII fields.

[0083] Fig. 6A 6 is a flow chart of an example method 600A that can be implemented in a customer data source (e.g., IP data source 302) for providing IP data to a secure connector in plain text. At box 601, the customer data source generates a DEK and a KEK locally using a cloud KMS. The customer data source encrypts the DEK using the KEK using an API of the cloud KMSI. At box 602, the customer data source performs authentication with the secure connector. The security credential service can then configure the cloud KMS to enable the KEK to be decrypted at the secure connector and the PII matching module. At box 620, the customer data source provides the encrypted DEK to the secure connector, and at box 630, provides the data to the secure connector in plain text over a secure link (see Fig. 7A , stage 702A or FIG. C, stage 702C).

[0084] Figure 6B is roughly similar to Fig. 6A Flow chart of example method 600B. However, here the customer data source encrypts the PII field at box 622 (see Figure 7B , stage 702B), and providing the data set to a secure connector over an encrypted link.

[0085] Additional considerations

[0086] The following additional considerations apply to the foregoing discussion.

[0087] The client device (e.g., client device 102) in which the techniques of the present disclosure may be implemented may be any suitable device capable of wireless communication, such as a smartphone, a tablet computer, a laptop computer, a desktop computer, a mobile game console, a point of sale (POS) terminal, a health monitoring device, a drone, a camera, a media streaming dongle or another personal media device, a wearable device such as a smart watch, a wireless hotspot, a femtocell, or a broadband router. In addition, in some cases, the client device may be embedded in an electronic system such as a head unit of a vehicle or an advanced driver assistance system (ADAS). Further, the client device may operate as an Internet of Things (IoT) device or a mobile Internet device (MID). Depending on the type, the client device may include one or more general purpose processors, a computer readable memory, a user interface, one or more network interfaces, one or more sensors, and the like.

[0088] Certain embodiments are described in the present disclosure as including logic or multiple components or modules. A module may be a software module (e.g., code stored on a non-transitory machine-readable medium) or a hardware module. A hardware module is a tangible unit that is capable of performing certain operations and may be configured or arranged in a particular manner. A hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations (e.g., as a dedicated processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations (e.g., as contained in a general-purpose processor or other programmable processor). The decision to implement a hardware module with a dedicated and permanently configured circuitry or with a temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

[0089] When implemented in software, the techniques may be provided as part of an operating system, as a library used by multiple applications, as a specific software application, etc. The software may be executed by one or more general-purpose processors or one or more special-purpose processors.

Claims

1. A method for performing a federated operation in one or more servers, the method comprising: receiving, at a module executing in a trusted execution environment (TEE), a first data set including personally identifiable information (PII) data and non-PII data from a first party (IP) data source; preprocessing the PII data to generate first formatted PII data, wherein the first formatted PII data conforms to a predefined format; matching, in the TEE, the first formatted PII data with second formatted PII data included in a second data set; performing a join operation between the first data set and the second data set based on the match to generate a joined data set; as well as The federated data set is provided to a data service that operates independently of the IP data source.

2. The method of claim 1, further comprising: Authentication with said IP data source is performed by said module and prior to receiving said first data set.

3. The method of claim 2, wherein performing the authentication comprises performing a decryption operation using credentials associated with the IP data source.

4. The method according to claim 1 or 2, wherein: The module implements a secure connector configured to use credentials associated with the IP data source; and The match is implemented in a security federation module, which is blocked from accessing the credentials associated with the IP data source.

5. The method of claim 4, further comprising: The first formatted PII data is provided from the secure connector to the security federation module via an extract-transform-load (ETL) pipeline.

6. The method of claim 5, wherein the ETL pipeline is configured to provide the first formatted PII data to (i) the data service and (ii) a repository for time-delay consumption of the first formatted PII data.

7. The method of any one of claims 4 to 6, further comprising: receiving, at the secure connector, an encrypted DEK associated with the IP data source; The encrypted DEK is provided from the secure connector to the security association module.

8. The method of claim 7, further comprising: Decrypting the encrypted DEK using a key management service (KMS) to generate a DEK; in: The PII data received from the IP data source together with the first data set is encrypted; The pre-processing of the PII data includes decrypting the received PII data before generating the first formatted PII data.

9. The method of claim 8, further comprising: The first formatted PII data is encrypted at the secure connector using the DEK before providing the first formatted PII data to the secure federation module.

10. The method of claim 8, further comprising: The first formatted PII data is hashed at the secure connector before providing the first formatted PII data to the secure federation module.

11. The method of claim 10, wherein the matching of the first formatted PII data with the second formatted PII data is implemented in the security federation module and is based on hashed PII data.

12. The method of claim 11, further comprising: Non-matching rows in the first data set and the second data set are discarded.

13. The method of any one of claims 1 to 3, wherein: The receiving and the matching are both implemented in a module configured to operate as a secure connector and a PII matching component; and Providing the federated data set includes providing the federated data set from the secure connector and the PII matching component to the data service via an ETL pipeline.

14. The method of any preceding claim, wherein receiving the first data set comprises receiving the first data set in clear text over an encrypted link.

15. A system comprising: One or more servers comprising processing hardware and configured to implement a method according to any one of the preceding claims.

Images